There are similar questions about related problems that I honestly didn't understand. I believe this might be a lack of statistics knowledge of my part. Quantization error standard deviation Error analysis for uniform quantization of uniform input Signal to Quantization Noise ratio concept

I was reading David R. Smith's Digital Transmission Systems and in it I found the following derivation for the average error due to quantization.

enter image description here

I don't understand going from 3.8 to 3.9 It might be a math problem on my end.

The last two integrals will disappear, because they correspond to the overload regions of the quantizer, and we are assuming that the input signal, x, does not have values in that region, meaning, in the last 2 integrals $p(x)=0$ Thus, they are irrelevant.

Looking at just the first integral (ignoring the summation, because it doesn't bother me), for those that can't see the picture we have

$$ \int_{x_{j-1}}^{x_j} (y_j - x)^2 p(x) dx $$

$x_j$ and $x_{j-1}$ correspond to input values of the quantizer that are the limits of a region in which the output of the quantizer is ${y_j}$. For example, if the input of the quantizer is between 2V to 3V let's say the output of the quantizer is 2.5V. $x_j$ for this case is 3V,$x_{j-1}$ is 2V and {y_j} is 2.5V $p(x)$ is just the probability that the input signal takes on the value $x$, so $p(2.3)$ is the probability that the input signal takes on the value 2.3V

Now for the following step we assume that $p(x)$ is a constant equal to $p(y_j)$ That I understood, but I can't understand how (I removed the constant probability from the integral to simplify) $$ \int_{x_{j-1}}^{x_j} (y_j - x)^2 dx $$ turned into $$ \frac{(x_j-x_{j-1})^2}{12} * (x_j-x_{j-1}) $$

Where did the $y_j$ go?

  • 1
    $\begingroup$ It's a good question. This treatment is not very good. There needs to be an explicit relation that says something like $$ y_j = (x_j - x_{j-1}) \left\lfloor \frac{x}{x_j - x_{j-1}} \right\rfloor $$ where $\lfloor \cdot \rfloor$ is the floor( ) function (rounds down to the nearest integer). $\endgroup$ May 10 at 19:01
  • $\begingroup$ @robertbristow-johnson thank you so much! I was feeling very stupid :( $\endgroup$ May 10 at 21:49

1 Answer 1




you obtain


  • $\begingroup$ Oh, thank you! I did not find that assumption in the text. Rereading it I didn't find it again. I understand how it makes sense though. $\endgroup$ May 11 at 18:16
  • 1
    $\begingroup$ @chilliefiber: I totally agree that this should've been mentioned somewhere in the book. $\endgroup$
    – Matt L.
    May 12 at 10:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.