# Quantization error standard deviation

Here I found that "pure" quantization error standard deviation of the signal is $$1/\sqrt{12}$$ of LSB. Where does it come from?

In the case of uniform quantization, and under some light hypothesis for the signal, the error can be modeled as an additive IID signal, independent of the signal, and with uniform distribution between +/- half LSB.

The power of such error is then $$\Delta^2/12$$, where $$\Delta$$ is the amplitude of one LSB.

Taking the square root and calling it standard deviation is something I'd never do, since standard deviation is used when approximating distributions as gaussian, which is not the case here. I'd stick to power as given above.

P.S. The result $$\Delta^2/12$$ is the variance of a uniformly distributed random variable: $$\int_{-\Delta/2}^{\Delta/2}(1/\Delta)x^2dx$$.

• why do divide by exactly $12$? Nov 26, 2021 at 17:16
• thank you for edit, now it is clear! Nov 26, 2021 at 17:50
• It's OK to calculate and use standard deviation for this. Standard deviation is not restricted to Gaussian distributions. Nov 28, 2021 at 0:08

The author is modeling quantization noise as being white (i.e., each sample is independent of previous or following samples) with each sample being a zero-mean, uniformly distributed random number with a span of one: $$p(x) = \begin{cases}1 & -\frac{1}{2} < x < \frac{1}{2} \\ 0 & \mathrm{otherwise}\end{cases}. \tag 1$$

Do the math (or look it up), and you'll find that the standard deviation of the PDF in (1) is $$\sqrt{\frac{1}{12}}$$.

Note that you can't always count on quantization acting like that. Quantization, in and of itself is a purely deterministic phenomenon. Moreover, because it involves a stair-step function, it can be mathematically vicious to try to model accurately.

A useful way of modeling nonlinearities -- if it works -- is to model them as some ideal linear process with injected noise. This is what's being done with quantization. However note that modeling the noise as white and uniformly distributed only works if the input to your ADC is much larger than a quantization step and has a tendency toward randomness.

Fortunately for a lot of DSP work that condition is met: the conversion stages of ADCs (the R-2R or C-2C ladders and associated analog switches) are easier to make than the front end (buffer and comparator), so most high bit-count ADCs have enough conversion bits included so that the quantization noise is significantly less than the random noise in the front end. This means that the quantization noise can be accurately modeled as uniformly distributed and white.

Deeper in the signal processing chain, this condition can be easily met, too. In the process of filtering a signal one often truncates the result, reducing the precision and generating quantization noise. As long as the input to this quantization is varying and large enough, the result can be modeled as adding random noise with uniform distribution.

However (and I've run into this in control systems work) it's possible to get an ADC where the quantization is bigger than other signals in the system, or to screw up the signal processing and create a quantization step with the same effect as a low-noise, large-step ADC. In that case you need to choose a more pessimistic "additive noise" model, or (in closed loop, which includes IIR filters) you need to model the system with a built-in step discontinuity (around which your feedback loop will, inevitably oscillate).

So if you can't assure yourself that the quantization noise isn't subsumed in signal, you need to either choose a model where it is a square wave at the worst possible frequency, or just throw up your hands in despair and simulate it. (Or add more bits to your data path, which is to be vastly preferred if you have the computational resources).

• However, Tim, the quantization that i usually worry about is not that coming from an ADC, but is simply word-length reduction for storing samples into states for an IIR (or some other alg with states) or for the output for nearly any alg. You do a bunch of calculations in double-precision float (or fixed) and then you have to send off a result in a 16-bit or 24-bit fixed (or even a float), and that word-length reduction can be well modeled as this uniform p.d.f. pseudorandom number with variance of $\frac{\Delta^2}{12}$. with float, it's more complicated, but it can be modeled. Nov 26, 2021 at 18:42
• I clarified my text based on your comment. I do think that the notion that quantization can end up being more severe than pseudorandom uniform noise, because if you aren't thinking about the possibility, you aren't going to think to avoid it. Nov 26, 2021 at 20:37