Improving Digitization by Adding White Noise Before Oversampling

I was reading a physics book and it said that when digitizing a signal white noise was added to improve the process. I don't understand how this works. Therefore my question is:

When digitizing an analogue signal, how can adding noise be beneficial when combined with oversampling in time?

• Are you talking about dithering? See en.wikipedia.org/wiki/Dither for more.
– alarge
Dec 29 '14 at 15:36
• @DavidZ If this off topic why are there already so many answers? Several people have said in the meta that they like to keep math questions as long as they're relevant to physics. Why should that thought process differ for other topics? Dec 29 '14 at 16:07
• @DanielSank because a question being off topic doesn't stop people from answering it. If you think signal processing questions should be on topic, and you think the community agrees with you, take it to meta and make a post proposing that change to the site's scope. Dec 29 '14 at 17:29
• Experimentalists do have to deal to some extent with the vagaries of digital sampling. I don't exactly agree with moving a question from a graduated site (physics stackexchange) to a beta site (this site), but what's done is done. Hopefully this migration will garner attention from people with greater knowledge. My knowledge with regard to clean signals is pre-millennial. The last time I had to work with real-world signals I didn't just have a low signal-to-noise problem. I had an over-unity noise-to-signal problem. Dealing with that was quite fun (in a masochistic sense). Dec 29 '14 at 20:11

Adding uncorrelated (i.e. white) noise to an analog signal prior to digitization is called dithering. To understand why we would do this we need to understand the idea of quantization noise. Consider an analog system which has signals ranging in amplitude from 0 to 100. Suppose we digitize this signal with a digitizer whose digital levels are spaced by 1. In other words, the possible digitized levels are

$$\left\{ -100, -99, -98 \ldots 99, 100 \right\} \, .$$

Now suppose the analog signal $s(t)$ is a DC signal of value 0.8, in other words

$$s(t) = 0.8 \, .$$

If we put this into the digitizer, the digitizer will round it to 1 and our digital samples $s_n$ will be

$$s_n = 1\,.$$

This is not good because now the our digital signal accumulates error as we acquire more signal. The digitized level is always too high so the longer we average the signal the more we over-estimate the analog level.

Adding white noise helps fix this problem because it pushes the analog level around such that it crosses neighboring digitization levels. Therefore, as you average over a set of digitized values you actually get something which is close to the true analog level. Let's see this via example.

Suppose the noise we add is Gaussian distributed with $\sigma=2$. Then the distribution of the analog signal is

$$p(x) \propto \exp \left[ \frac{-(x - s(t))^2}{2\sigma^2} \right] = \exp \left[ \frac{-(x-0.8)^2}{8} \right] \, .$$

We now compute the average digital value $\langle s_n \rangle$ by averaging $p(x)$ over the integers. The normalization constant for the distribution is $$N = \sum_{m=-100}^{100} \exp \left[ \frac{-(m-0.8)^2}{8} \right]$$ and so we have $$\langle s_n \rangle = \frac{1}{N} \sum_{m=-100}^{100} m \exp\left[ \frac{-(m-0.8)^2}{8}\right] = 0.79999\ldots$$ Thus you can see that adding the white noise caused the average digitized signal to more closely match the true analog value.

Of course, adding the noise makes your signal to noise ratio worse. That means to to actually have a high probability of measuring the $\langle s_n\rangle$ we just computed, you have to take more samples than you would think in the noiseless case. This is why you hear about over-sampling and dithering at the same time. Because the dithering noise is truly uncorrelated, taking more samples always helps improve the signal to noise ratio, even if you're sampling way above the bandwidth of the analog signal coming in.

Yes, adding uncorrelated noise to a signal, then averaging multiple samples, is one way of dealing with quantization noise.

To make this obvious, think of the limiting case. You want to measure the value of a signal between 0 and 1, but all you have is a 1-bit A/D. The A/D output is 0 when the signal is from 0 to .5, and 1 when the signal is .5 to 1.

Now consider a signal that is .3. With a nice clean .3 in, the A/D will always yield 0. You can't distinguish between a input signal of .1, .25, .3, etc, for example.

Now add random noise of ±.5. The signal seen by the A/D will now be somewhere from -.2 to .8. A 0 output has a probability of .7, and a 1 output of .3. Any one reading won't tell you much (unlike before when it told you the signal was from 0 to .5), but after a bunch of readings you get a reasonable idea of the signal. For example, if you take 100 readings, around 30 of them will be 1 and 70 will be 0. There is no guarantee of this, but the more readings you average, the higher the confidence that the average result represents the signal.

So dithering is a means of trading off certainty and bandwidth to get resolution. Note that no matter how many readings you take, you don't know the signal is within some bound, only that the probability of it being within some bound is high. Also note that you have lost bandwidth since many samples are required to get one signal value. Nonetheless, in many cases these are useful tradeoffs.

Dithering (intentionally adding noise) the input signal combined with oversampling can (no guarantees!) improve the effective number of bits in the signal and increase the signal to noise ratio.

Assuming the underlying process is a white noise process, the digitized signal will still look like white noise if the quantization level is well into the noise. This is in general a good thing. White noise has rather nice mathematical properties. Dealing with the non-linear, signal-dependent quantization noise is difficult. That quantization noise can generally be ignored if the quantization level is well into the noise.

What if the quantization level isn't well into the noise? Now you have messy quantization noise to deal with. One way to deal with it is to use a higher resolution ADC and make the quantization be well into noise. An alternative is to intentionally add white noise to the input signal so as to make the quantization noise from your cheap ADC small compared to the (dithered) signal noise.

The basic idea is to dither the input signal so that the noise in the dithered signal dominates over quantization noise (but obviously not by too much). The dithered signal is intentionally oversampled and then downsampled by averaging two or more successive measurements. This averaging only buys something if the input signal is noisy compared to the quantization noise. Averaged quantization noise looks like quantization noise. Averaging white noise lets signal rise out of the noise.

For more, read Walt Kester's tutorial The Good, the Bad, and the Ugly Aspects of ADC Input Noise—Is No Noise Good Noise?.