# Design a quantization scheme around a desired quantization noise distribution

I have a - perhaps naive - question regarding the quantization noise/error. Assuming the goal is not the performance of the quantizer but rather being able to model the quantization noise exactly - say, Gaussian with certain mean and variance - are there any methods that would let one achieve this? There are no assumptions about the input distribution, it is most likely sampled from a mixture of different distributions.

Edit for further explanation:

My intention is to quantize a dataset, e.g. a set of images or general tensors. I have found out that the quantization noise is correlated with input distribution which gets in the way of modelling it independently. If one uses dithering, then AFAIK the noise is decoupled from the input distribution. But still the question remains, can I design a quantization scheme, e.g. lattice quantization, so that I can derive the exact mean/std of the Gaussian noise pdf?

• While I disagree with "ban" on crossposting, I do think a day should pass to give answering a chance. Feb 15 at 13:25
• Thanks for the feedback. I am very new to this community. I posted it on electronics.stackexchange but then realized this is a better place to ask the question. I didn't know about any bans on cross posting. Should I remove my original post from "electronics"? Feb 15 at 13:33
• Yes, you should post where it fits best. The ban isn't exactly "official", enforcement varies across networks, but it's much less tolerated to post the same thing in many places at the same time. Feb 15 at 13:39
• Thanks! I'll remove the other one Feb 15 at 13:40
• " I have found out that the quantization noise is correlated with input distribution" What exactly is correlated ? The Q noise is often correlated with the input signal itself but being correlated with the distribution of the input signal is really odd. A signal and it's distribution are VERY different things. Can you show some example data? Feb 15 at 20:31

Let's get some terminology in place first.

For simplicity I will use time discrete signals but it's the same for time continous signals as well.

We assume an amplitude-continuous signal $$x[n]$$ that is turned into a amplitude discrete signal $$y[n]$$ using the quantization process $$Q$$, so we have

$$y[n] = Q\{x[n]\}$$

The quantization noise $$q[n]$$ is simply the difference

$$q[n] = y[n]-x[n]$$

The quantization process is time invariant if the process quantize the signal always the same way, or $$Q\{x[n-N]\} = y[n-N]$$. Most quantizers are time invariant simply because they want to minimize the quantization error and "round to the nearest" does exactly that.

For a time-invariant quantizer quantization noise and signal are highly correlated. A simple example: if the signal is periodic the quantization noise will also be periodic with the same period. The standard way of decorrelating noise and signal is to add a dither. This breaks the time invariance of the quantizer but also adds quantization noise. Dither design is always a trade off.

For a time invariant quantizer the probability density function (PDF) of the quantization noise can be directly calculated from the PDF of the input signal and the quantization process. The PDF is the first derivative of the Distribution Function.

With this out of the way, we can start diving into the questions

I have found out that the quantization noise is correlated with input distribution

I don't think that's the case. The quantization noise is corelated to the input signal, but not to its distribution. Signals can be uncorrelated and even if they have the same distribution and correlated signals can have totally different distributions. I think you are confusing two different concepts here.

If one uses dithering, then AFAIK the noise is decoupled from the input distribution.

Again, no. Dithering de-correlates the quantization noise from the input signal but that has nothing to do with either distribution.

can I design a quantization scheme, e.g. lattice quantization, so that I can derive the exact mean/std of the Gaussian noise pdf?

I think what you are asking here is "can I design a quantization process so that the PDF of the quantization noise is always the same regardless of the input signal's distribution".

The answer to that one is "not really". The PDF of the quantization noise is a function of the PDF of the input signal. Consider the example of a binary input signal: for a time invariant quantizer the quantization noise will also be binary, no matter how you quantize it. You can soften this up with a dither, but in order to get Gaussian quantization noise from a binary input signal the dither would basically have to drown the input signal completely.

Things get a little easier if you can make simplifying assumptions. For example if we assume a uniform quantizer, a reasonably "smooth" PDF of the input signal and a large number of quantization steps we can derive that the quantization noise is uniformly distributed over the quantization interval and that this is independent of the inputs PDF.

So I guess the final answer here is "maybe". It really depends on how you can constrain the problem and what assumptions you can make. If you want Gaussian quantization noise, you can try a uniform quantizer with a Gaussian distributed dither, play around with the level the dither and see of there is a sweet spot that works for you.