2
$\begingroup$

I was reading Simon S. Haykin's Digital Communications in order to understand the concept of quantization. However, on SQNR, I got stuck over the point where the author mentioned:

With the input $M$ having zero mean and the quantizer assumed to be symmetric, it follows that the quantizer output $V$ and, therefore, the quantization error $Q$ will also have zero mean. Thus, for a partial statistical characterization of the quantizer in terms of output signal-to-(quantization) noise ratio, we need only find the mean-square value of the quantization error $Q$.

where $M$ is the random variable of the sampled analog inputs, $V$ is the random variable of digitalized outputs and $Q$ is the random variable of the error.

$\star$ Can someone kindly elaborate this line like how does zero mean of $M$, $V$ and $Q$ implicates that we need to find the mean-square value?

I searched for the meaning of zero mean and partial statistical characterization and got them as follows:

https://www.sciencedirect.com/topics/engineering/statistical-characterization

Mean and correlation provide a partial statistical characterization of a random process in terms of its averages and moments. They are useful as they offer tractable mathematical analysis, they are amenable to experimental evaluation and they are well suited to the characterization of linear operations on random processes.

and Zero mean : https://physics.stackexchange.com/questions/178323/what-does-zero-mean-random-noise-with-standard-deviation-equal-to-1-mean

I just don't know how these concepts come together to imply that we need to find mean square value. Please help!

$\endgroup$
3
  • 3
    $\begingroup$ If the mean is zero, then the mean-square value corresponds to the variance. $\endgroup$ Oct 3 at 16:07
  • $\begingroup$ What Haykin is saying is not quite right in ever so many ways. $\endgroup$ Oct 3 at 20:33
  • $\begingroup$ @CrisLuengo , yeah that's one understanding but what Dilip Sarwate sir says also seems valid. I am just a neophyte in this subject and this book is considered to be "text books" for undergraduates. So it will be difficult for me to find out the fallacy. Can you please clarify further considering Dilip sir's points? $\endgroup$ Oct 4 at 0:48
6
$\begingroup$

Give me an A! Give me a D! Give me a converter! What have we got? An A/D converter! Go Team!

Let $X$ denote a standard Gaussian random variable with pdf $\phi(x)$ and complementary CDF $Q(x)$. Let $Y$ denote a quantized version of $X$. First, for simplicity (and to flex our analytical muscles), suppose that $$Y = \begin{cases}+\alpha, & X \geq 0,\\ -\alpha, & X < 0.\end{cases}$$ Note that $Y$ is a discrete random variable taking on values $\pm\alpha$ with equal probability $\frac 12$.

If we use $Y$ as a quantized representation of $X$, then the quantization error or quantization noise is $$Z = X-Y = \begin{cases}X-\alpha, & X \geq 0,\\ X+\alpha, & X < 0.\end{cases}$$ Note that $Z$ takes on values in $[-\alpha,\infty)$ when $X\geq 0$ and values in $(-\infty,\alpha)$ when $X < 0$. Furthermore, the mean-square error of our representation is $E[Z^2] = E[(X-Y)^2]$ which we can calculate as \begin{align} E[Z^2] &= \int_0^\infty (x-\alpha)^2\phi(x)\, \mathrm dx + \int_{-\infty}^0 (x+\alpha)^2\phi(x)\, \mathrm dx\\ &= \int_{-\infty}^\infty (x^2+\alpha^2)\phi(x)\, \mathrm dx -4\alpha\int_0^\infty x \phi(x)\, \mathrm dx\\ &= 1 + \alpha^2 - 2\sqrt{\frac{2}{\pi}}\alpha. \end{align} Note that in the last step, we used the fact that the antiderivative of $x\phi(x)$ is $-\phi(x)$ to evaluate the second integral. From this, we get that the smallest possible value of $E[Z^2]$ is $1 - \frac{2}{\pi}$ and it occurs when we choose $\alpha = \sqrt{\frac{2}{\pi}}$.

Next, suppose that we quantize $X$ into $7$ integer values from $-3$ to $+3$, mapping the observed value into the nearest of these $7$ integers. Thus, $W$ is a discrete random variable and its pmf is readily calculated as \begin{align} p_W(3) &= p_W(-3) = Q(2.5) &= 0.0006,\\ p_W(2) &= p_W(-2) = Q(1.5)-Q(2.5) &= 0.0606,\\ p_W(1) &= p_W(-1) = Q(0.5)-Q(1.5) &= 0.2417,\\ p_W(0) &= Q(-0.5)-Q(0.5) &= 0.3830. \end{align} The mean-square error can be worked out using the ideas and methods described above. More fun can be had by using quantization levels ranging from $-3\alpha$ to $+3\alpha$ and then figuring out the optimum choice of $\alpha$ that minimizes the mean-square error. Alternatively, quantize $X$ into $8$ levels $\pm\frac 12, \pm\frac 32, \pm\frac 52, \pm\frac 72$ and do the calculations as above.

But, turning back to integer levels $W$, if $W$ is represented as a $3$-bit twos complement number $[W_2, W_1, W_0]$, then the $W_i$ are Bernoulli random variables with parameters $p_2 =P(W<0) = 0.3085$, $p_1 = 0.3691$ and $p_0 = 0.4958$. Note that $[W_2,W_1, W_0]$ cannot take on value $100$ which is $-4$ in three-bit twos' complement notation. Finally, we are at the A/D converter promised in the blurb that begins this answer,

How all this fits in with the various claims in Haykin's book is left as an exercise for the reader.

$\endgroup$
1
  • $\begingroup$ @ Dilip Sarwate , Sir, in the book, Haykin is considering uniform pdf ranging from (- delta /2) to (+ delta /2) and we know that expectation or mean value of such uniform pdf lies at X(random variable)=0 and that's how the author is coming into such a conclusion. I got to know this now. It was my mistake, really sorry for the confusion. And really thankful for this answer as it lets us know about the entire A/D conversion of standard Gaussian random variable. Generally, it's difficult to be found in books provided to undergrads. $\endgroup$ Oct 12 at 1:58
5
$\begingroup$

Thus, for a partial statistical characterization of the quantizer in terms of output signal-to-(quantization) noise ratio, we need only find the mean-square value of the quantization error Q.

All that stuff does not imply that you need to find the mean-square value. This may just be an English a usage thing -- in most contexts, "we need only" means "we only need to".

So what the author is saying is that, while there may be other ways of characterizing quantization noise, if you're willing to settle for a partial and statistical characterization (instead of some fully-realized nonlinear representation), then all you need to do is find the mean-square value of the quantization error.

This is one of the tricks in your bag of tricks to analyze nonlinear systems. Nonlinear systems analysis usually ranges from hard to impossible, but if you can approximate a nonlinear system as a linear system (which is easy to analyze) with some injected noise in an easy-to-analyze form (say, white Gaussian) noise, then you can analyze that approximation using known tools.

This is exactly what the quantization noise model of quantization does. Absolutely strictly speaking, quantization noise is utter fiction. However, approximating the effect of quantization as the sum of the input signal and quantization noise is very accurate for a number of situations -- so we use it, because the analysis becomes tractable, and the result accurately predicts how our system will behave.

$\endgroup$
1
  • $\begingroup$ sir In the book, Haykin is considering uniform pdf ranging from (- delta /2) to (+ delta /2) and we know that expectation or mean value of such uniform pdf lies at X(random variable)=0 and that's how the author is coming into such a conclusion. I got to know this now. It was my mistake, really sorry for the confusion. $\endgroup$ Oct 12 at 2:01
3
$\begingroup$

With the input M having zero mean and the quantizer assumed to be symmetric, it follows that the quantizer output V and, therefore, the quantization error Q will also have zero mean.

That doesn't seem to be correct unless there are additional assumptions. It's easy enough to construct a counter example: Let's assume the quantization step is $1$ and the unquantized sequence is M = [2.5 2.5 2.5 2.5 -2 -2 -2 -2 -2]. That's clearly zero mean.

We assume that the quantizer round 2.5 down to 2 (i.e. we treat 2.5 same as 2.499999) This quantizes M = [+2 +2 +2 +2 -2 -2 -2 -2 -2] . The mean is not zero, it's $-2/9$ So it's not zero mean.

The quantization error is also all negative so Q is not zero mean either.

$\endgroup$
5
  • $\begingroup$ Yes getting your point. But then how will we evaluate the expression for SQNR if the consideration itself is incorrect as this consideration given by the author seems to be the basis of evaluation of the SQNR? Can you please provide the complete answer? $\endgroup$ Oct 4 at 1:10
  • 2
    $\begingroup$ I’m not defending the author of that book, I know little of this subject. But it seems to me that this example is contrived, and in general, for a natural signal with a zero mean, the expected value of the mean of the discretized signal is zero. It is likely not exactly zero, but the expected value is zero. $\endgroup$ Oct 4 at 2:01
  • 3
    $\begingroup$ @CrisLuengo: of course it's contrived, that's the whole point. IMO a book should carefully distinguish between "this is always true, based on a mathematical proof" and "this is mostly true, but only if certain conditions/assumptions are being met". Getting these two confused and not checking the underlying assumption can result in multi-million dollar mistakes. $\endgroup$
    – Hilmar
    Oct 4 at 12:19
  • $\begingroup$ I agree with that! $\endgroup$ Oct 4 at 12:39
  • $\begingroup$ @CrisLuengo , sir in the book, Haykin is considering uniform pdf ranging from (- delta /2) to (+ delta /2) and we know that expectation or mean value of such uniform pdf lies at X(random variable)=0 and that's how the author is coming into such a conclusion. I got to know this now. It was my mistake, really sorry for the confusion. $\endgroup$ Oct 12 at 2:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.