I am currently studying source coding of waveform in a course of Information Theory and Coding. More precisely, I am trying to understand, both mathematically and intuitively, transform coding and its advantages of direct scalar quantization. I believe I am quite OK with the mathematical side. However, I am still lacking a bit of intuition.

My current understanding is based on the following sources:

In the case of a memoryless source and with scalar quantization under the high-resolution quantization assumption, the distortion-rate function is given by $$ D(R) = \epsilon^2 \cdot \sigma^2 \cdot 2^{-2R} $$ or, equivalently, the rate-distortion function is given by $$ R(D) = \frac{1}{2}\log_2\left(\epsilon^2\frac{\sigma^2}{D}\right)$$ where $D$ is the distortion (measured as a MSE), $R$ is the rate in bits/sample of the source, $\sigma^2$ is the variance of the source and $\epsilon$ is a factor that depends on the probability density function of the source and the properties of the quantizer (fixed-length vs. variable length). I am primarily interested with variable-length quantizers. For this particular case, optimal quantizers have uniform step sizes. Also, with the high-resolution quantization assumption, $D \approx \frac{\Delta^2}{12}$ with $\Delta$ the quantization step.

Intuitively, I understand that if the source is no longer memoryless, applying vector quantization on vector of $N$ samples from the source will exploit statistical dependencies of the source (this is called memory advantage of VQ in the aforementioned slides from Berlin IC). I also understand that vector quantization is problematic from a complexity point of view. Then here comes transform coding. Currently, the interpretation that most satisfies me is the one provided here, slide 14, repeated here for convenience.

Effect of an orthogonal transform

The orthogonal transform concentrates the energy in the first transform coefficient. It can also be seen that the number of quantization cells with appreciable probabilities is reduced. Because the average rate is equal to the entropy of the quantized samples, this indicates a rate reduction.

However, this intuition seems to be contradictory with the fact that $\epsilon^2$ in the rate-distortion equation is for example, equal to $1.25$ for a source with Laplacian distribution and $1.0$ for a source with uniform distribution (again in the case of an optimal variable-length quantizer, that is with uniform quantization step $\Delta$). Indeed, for a Laplacian distribution, the entropy of the quantized samples should be reduced compared to the uniform distribution, and so the rate should also be reduced, which is not indicated by the rate-distortion equation. My hypothesis at this point is that $\epsilon^2 > 1$ (and so $R(D)$ is higher than in the uniform distribution case) to take into account the increase in distortion that should intuitively appear when using an uniform quantizer with a Laplacian distribution instead of a uniform distribution.

At the end of the day, I am a bit stuck by a lack of a clear intuitive understanding of transform coding and would be glad if someone could provide me with such an understanding and/or correct anything I said that is not correct/not precise.

  • 1
    $\begingroup$ May be the following books could help? 1-) Multiresolution Signal Decomposition_AKANSU, 2-Introduction to Data Compression_SAYOOD, 3-) Waveform Quantization and Coding_Jayant, 4-) Vector Quantization_GERSHO... In fact almost any book on Image & Video Compression should involve a discusssion of transform coding that would be highly intuitive I guess. $\endgroup$
    – Fat32
    May 30, 2017 at 22:58


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.