Discretization

 ( Numerical Analysis ) 

In applied mathematics, discretization is the process of transferring continuous functions, models, variables, and equations into counterparts. This process is usually carried out as a first step toward making them suitable for numerical evaluation and implementation on digital computers. Dichotomization is the special case of discretization in which the number of discrete classes is 2, which can approximate a continuous variable as a binary variable (creating a dichotomy for conceptual model purposes, as in binary classification).

Discretization is also related to discrete mathematics, and is an important component of granular computing. In this context, discretization may also refer to modification of variable or category granularity, as when multiple discrete variables are aggregated or multiple discrete categories fused.

Whenever continuous data is discretized, there is always some amount of discretization error. The goal is to reduce the amount to a level considered for the conceptual model purposes at hand.

The terms ''discretization '' and ''quantization'' often have the same [[denotation]] but not always identical [[connotations]]. (Specifically, the two terms share a [[semantic field]].) The same is true of discretization error and quantization error.
     

Mathematical methods relating to discretization include the Euler–Maruyama method and the zero-order hold.

---

Discretization of linear state space models Discretization is also concerned with the transformation of continuous differential equations into discrete difference equations, suitable for numerical computing.

The following continuous-time state space model

$$\backslash begin\{align\}$$

 \dot{\mathbf{x}}(t) &= \mathbf{Ax}(t) + \mathbf{Bu}(t) + \mathbf{w}(t) \\[2pt]
 \mathbf{y}(t) &= \mathbf{Cx}(t) + \mathbf{Du}(t) + \mathbf{v}(t)
     

\end{align}

where and are continuous zero-mean white noise sources with power spectral densities

$$\backslash begin\{align\}$$

 \mathbf{w}(t) &\sim N(0,\mathbf Q) \\[2pt]
 \mathbf{v}(t) &\sim N(0,\mathbf R)
     

\end{align}

can be discretized, assuming zero-order hold for the input and continuous integration for the noise , to

$$\backslash begin\{align\}$$

 \mathbf{x}[k+1] &= \mathbf{A_d x}[k] + \mathbf{B_d u}[k] + \mathbf{w}[k] \\[2pt]
 \mathbf{y}[k] &= \mathbf{C_d x}[k] + \mathbf{D_d u}[k] + \mathbf{v}[k]
     

\end{align}

with covariances

$$\backslash begin\{align\}$$

 \mathbf{w}[k] &\sim N(0,\mathbf{Q_d}) \\[2pt]
 \mathbf{v}[k] &\sim N(0,\mathbf{R_d})
     

\end{align}

where

 \mathbf{B_d} &= \left( \int_{\tau=0}^{T}e^{\mathbf A \tau}d\tau \right) \mathbf B \\[4pt]
 \mathbf{C_d} &= \mathbf C \\[8pt]
 \mathbf{D_d} &= \mathbf D \\[2pt]
 \mathbf{Q_d} &= \int_{\tau=0}^{T} e^{\mathbf A \tau} \mathbf Q e^{\mathbf A^\top \tau}  d\tau \\[2pt]
 \mathbf{R_d} &= \mathbf R \frac{1}{T}
     

\end{align}

and is the sample time. If is nonsingular, $\backslash mathbf\{B\_d\}\; =\; \backslash mathbf\; A^\{-1\}(\backslash mathbf\{A\_d\}\; -\; \backslash mathbf\{I\})\backslash mathbf\; B.$

The equation for the discretized measurement noise is a consequence of the continuous measurement noise being defined with a power spectral density.

A clever trick to compute and in one step is by utilizing the following property:Raymond DeCarlo: Linear Systems: A State Variable Approach with Numerical Implementation, Prentice Hall, NJ, 1989

$$$$

 e^{\begin{bmatrix}
   \mathbf{A} & \mathbf{B} \\
   \mathbf{0} & \mathbf{0}
 \end{bmatrix} T} = \begin{bmatrix}
   \mathbf{A_d} & \mathbf{B_d} \\
   \mathbf{0} & \mathbf{I}
 \end{bmatrix}
     

Where and are the discretized state-space matrices.

---

Discretization of process noise Numerical evaluation of is a bit trickier due to the matrix exponential integral. It can, however, be computed by first constructing a matrix, and computing the exponential of itCharles Van Loan: Computing integrals involving the matrix exponential, IEEE Transactions on Automatic Control. 23 (3): 395–404, 1978 $$\backslash begin\{align\}$$

 \mathbf{F} &= \begin{bmatrix}
   -\mathbf{A} & \mathbf{Q} \\
   \mathbf{0} & \mathbf{A}^\top
 \end{bmatrix} T \\[2pt]
 \mathbf{G} &= e^\mathbf{F} = \begin{bmatrix}
   \dots & \mathbf{A_d}^{-1}\mathbf{Q_d} \\
   \mathbf{0} & \mathbf{A_d}^\top
 \end{bmatrix}
     

\end{align} The discretized process noise is then evaluated by multiplying the transpose of the lower-right partition of with the upper-right partition of : $$\backslash mathbf\{Q\_d\}\; =\; (\backslash mathbf\{A\_d\}^\backslash top)^\backslash top\; (\backslash mathbf\{A\_d\}^\{-1\}\backslash mathbf\{Q\_d\})\; =\; \backslash mathbf\{A\_d\}\; (\backslash mathbf\{A\_d\}^\{-1\}\backslash mathbf\{Q\_d\}).$$

---

Derivation Starting with the continuous model $$\backslash mathbf\{\backslash dot\{x\}\}(t)\; =\; \backslash mathbf\{Ax\}(t)\; +\; \backslash mathbf\{Bu\}(t)$$ we know that the matrix exponential is $$\backslash frac\{d\}\{dt\}e^\{\backslash mathbf\{A\}t\}\; =\; \backslash mathbf\{A\}e^\{\backslash mathbf\{A\}t\}\; =\; e^\{\backslash mathbf\{A\}t\}\; \backslash mathbf\; A$$ and by premultiplying the model we get $$e^\{-\backslash mathbf\{A\}t\}\; \backslash mathbf\{\backslash dot\{x\}\}(t)\; =\; e^\{-\backslash mathbf\{A\}t\}\; \backslash mathbf\{Ax\}(t)\; +\; e^\{-\backslash mathbf\{A\}t\}\; \backslash mathbf\{Bu\}(t)$$ which we recognize as $$\backslash frac\{d\}\{dt\}\backslash Bigle^\{-\backslash mathbf\{A\}t\}\backslash mathbf\; =\; e^\{-\backslash mathbf\{A\}t\}\; \backslash mathbf\{Bu\}(t)$$ and by integrating, $$\backslash begin\{align\}$$

 e^{-\mathbf{A}t}\mathbf{x}(t) - e^0\mathbf{x}(0) &= \int_0^t e^{-\mathbf{A}\tau} \mathbf{Bu}(\tau) d\tau \\[2pt]
 \mathbf{x}(t) &= e^{\mathbf{A}t}\mathbf{x}(0) + \int_0^t e^{\mathbf{A}(t-\tau)} \mathbf{Bu}(\tau) d\tau
     

\end{align} which is an analytical solution to the continuous model.

Now we want to discretise the above expression. We assume that is constant during each timestep. $$\backslash begin\{align\}$$

 \mathbf x[k] &\, \stackrel{\mathrm{def}}{=}\ \mathbf x(kT) \\[6pt]
 \mathbf x[k] &= e^{\mathbf{A}kT}\mathbf x(0) + \int_0^{kT} e^{\mathbf A(kT-\tau)} \mathbf{Bu}(\tau) d\tau \\[4pt]
 \mathbf x[k+1] &= e^{\mathbf A(k+1)T}\mathbf x(0) + \int_0^{(k+1)T} e^{\mathbf A[(k+1)T-\tau]} \mathbf{Bu}(\tau) d \tau \\[2pt]
 \mathbf x[k+1] &= e^{\mathbf{A}T} \left[  e^{\mathbf{A}kT}\mathbf x(0) + \int_0^{kT} e^{\mathbf A(kT-\tau)} \mathbf{Bu}(\tau) d \tau \right]+ \int_{kT}^{(k+1)T} e^{\mathbf A(kT+T-\tau)} \mathbf B\mathbf u(\tau) d\tau
     

\end{align} We recognize the bracketed expression as $\backslash mathbf\; xk$, and the second term can be simplified by substituting with the function $v(\backslash tau)\; =\; kT\; +\; T\; -\; \backslash tau$. Note that $d\backslash tau=-dv$. We also assume that is constant during the integral, which in turn yields

$$\backslash begin\{align\}$$

 \mathbf x[k+1] &= e^{\mathbf{A}T}\mathbf x[k] - \left( \int_{v(kT)}^{v((k+1)T)} e^{\mathbf{A}v} dv \right) \mathbf{Bu}[k] \\[2pt]
 &= e^{\mathbf{A}T}\mathbf x[k] - \left( \int_T^0 e^{\mathbf{A}v} dv \right) \mathbf{Bu}[k] \\[2pt]
 &= e^{\mathbf{A}T}\mathbf x[k] + \left( \int_0^T e^{\mathbf{A}v} dv \right) \mathbf{Bu}[k] \\[4pt]
 &= e^{\mathbf{A}T}\mathbf x[k] + \mathbf A^{-1}\left(e^{\mathbf{A}T} - \mathbf I \right) \mathbf{Bu}[k]
     

\end{align}

which is an exact solution to the discretization problem.

When is singular, the latter expression can still be used by replacing $e^\{\backslash mathbf\{A\}T\}$ by its Taylor series, $$e^\{\backslash mathbf\{A\}T\}\; =\; \backslash sum\_\{k=0\}^\{\backslash infty\}\; \backslash frac\{1\}\{k!\}\; (\backslash mathbf\{A\}T)^k\; .$$ This yields $$\backslash begin\{align\}$$

 \mathbf x[k+1] &= e^{\mathbf{A}T}\mathbf x[k] + \left( \int_0^T e^{\mathbf{A}v} dv \right) \mathbf{Bu}[k] \\[2pt]
 &= \left(\sum_{k=0}^{\infty} \frac{1}{k!} (\mathbf{A}T)^k\right) \mathbf x[k] + \left(\sum_{k=1}^{\infty} \frac{1}{k!} \mathbf{A}^{k-1} T^k\right) \mathbf{Bu}[k],
     

\end{align} which is the form used in practice.

---

Approximations Exact discretization may sometimes be intractable due to the heavy matrix exponential and integral operations involved. It is much easier to calculate an approximate discrete model, based on that for small timesteps $e^\{\backslash mathbf\{A\}T\}\; \backslash approx\; \backslash mathbf\; I\; +\; \backslash mathbf\; A\; T$. The approximate solution then becomes: $$\backslash mathbf\; xk+1\; \backslash approx\; (\backslash mathbf\; I\; +\; \backslash mathbf\{A\}T)\; \backslash mathbf\; xk\; +\; T\; \backslash mathbf\{Bu\}k$$

This is also known as the Euler method, which is also known as the forward Euler method. Other possible approximations are $e^\{\backslash mathbf\{A\}T\}\; \backslash approx\; (\backslash mathbf\; I\; -\; \backslash mathbf\{A\}T)^\{-1\}$, otherwise known as the backward Euler method and $e^\{\backslash mathbf\{A\}T\}\; \backslash approx\; (\backslash mathbf\; I\; +\backslash tfrac\{1\}\{2\}\; \backslash mathbf\{A\}T)\; (\backslash mathbf\; I\; -\; \backslash tfrac\{1\}\{2\}\; \backslash mathbf\{A\}T)^\{-1\}$, which is known as the bilinear transform, or Tustin transform. Each of these approximations has different stability properties. The bilinear transform preserves the instability of the continuous-time system.

---

Discretization of continuous features In statistics and machine learning, discretization refers to the process of converting continuous features or variables to discretized or nominal features. This can be useful when creating probability mass functions.

---

Discretization of smooth functions In generalized functions theory, discretization arises as a particular case of the Convolution Theorem on tempered distributions

$\backslash mathcal\{F\}\backslash \{f*\backslash operatorname\{III\}\backslash \}\; =\; \backslash mathcal\{F\}\backslash \{f\backslash \}\; \backslash cdot\; \backslash operatorname\{III\}$

$\backslash mathcal\{F\}\backslash \{\backslash alpha\; \backslash cdot\; \backslash operatorname\{III\}\backslash \}=\; \backslash mathcal\{F\}\backslash \{\backslash alpha\backslash \}*\backslash operatorname\{III\}$

where $\backslash operatorname\{III\}$ is the Dirac comb, $\backslash cdot\; \backslash operatorname\{III\}$ is discretization, $*\; \backslash operatorname\{III\}$ is periodization, $f$ is a rapidly decreasing tempered distribution (e.g. a Dirac delta function $\backslash delta$ or any other compactly supported function), $\backslash alpha$ is a Smoothness, slowly growing ordinary function (e.g. the function that is constantly $1$ or any other Bandlimiting function) and $\backslash mathcal\{F\}$ is the (unitary, ordinary frequency) Fourier transform. Functions $\backslash alpha$ which are not smooth can be made smooth using a mollifier prior to discretization.

As an example, discretization of the function that is constantly $1$ yields the sequence $..,1,1,1,..$ which, interpreted as the coefficients of a linear combination of Dirac delta functions, forms a Dirac comb. If additionally truncation is applied, one obtains finite sequences, e.g. $1,1,1,1$. They are discrete in both, time and frequency.

---

See also

• Discrete event simulation
• Discrete space
• Discrete time and continuous time
• Finite difference method
• Finite volume method for unsteady flow
• Interpolation
• Smoothing
• Stochastic simulation
• Time-scale calculus

---

Further reading

• Robert Grover Brown & Patrick Y. C. Hwang (1997). 9780471128397 ISBN 9780471128397
• Chi-Tsong Chen (1984). 9780030716911, Saunders College Publishing. ISBN 9780030716911
• R.H. Middleton & G.C. Goodwin (1990). 9780132116657, Prentice Hall. ISBN 9780132116657

---

External links

• Discretization in Geometry and Dynamics: research on the discretization of differential geometry and dynamics

Post Comment