# Improving Digitization by Adding White Noise Before Oversampling

I was reading a physics book and it said that when digitizing a signal white noise was added to improve the process. I don't understand how this works. Therefore my question is:

When digitizing an analogue signal, how can adding noise be beneficial when combined with oversampling in time?

• Are you talking about dithering? See en.wikipedia.org/wiki/Dither for more. – alarge Dec 29 '14 at 15:36
• @DavidZ If this off topic why are there already so many answers? Several people have said in the meta that they like to keep math questions as long as they're relevant to physics. Why should that thought process differ for other topics? – DanielSank Dec 29 '14 at 16:07
• @DanielSank because a question being off topic doesn't stop people from answering it. If you think signal processing questions should be on topic, and you think the community agrees with you, take it to meta and make a post proposing that change to the site's scope. – David Z Dec 29 '14 at 17:29
• Experimentalists do have to deal to some extent with the vagaries of digital sampling. I don't exactly agree with moving a question from a graduated site (physics stackexchange) to a beta site (this site), but what's done is done. Hopefully this migration will garner attention from people with greater knowledge. My knowledge with regard to clean signals is pre-millennial. The last time I had to work with real-world signals I didn't just have a low signal-to-noise problem. I had an over-unity noise-to-signal problem. Dealing with that was quite fun (in a masochistic sense). – David Hammen Dec 29 '14 at 20:11

Adding uncorrelated (i.e. white) noise to an analog signal prior to digitization is called dithering. To understand why we would do this we need to understand the idea of quantization noise. Consider an analog system which has signals ranging in amplitude from 0 to 100. Suppose we digitize this signal with a digitizer whose digital levels are spaced by 1. In other words, the possible digitized levels are

$$\left\{ -100, -99, -98 \ldots 99, 100 \right\} \, .$$

Now suppose the analog signal $s(t)$ is a DC signal of value 0.8, in other words

$$s(t) = 0.8 \, .$$

If we put this into the digitizer, the digitizer will round it to 1 and our digital samples $s_n$ will be

$$s_n = 1\,.$$

This is not good because now the our digital signal accumulates error as we acquire more signal. The digitized level is always too high so the longer we average the signal the more we over-estimate the analog level.

Adding white noise helps fix this problem because it pushes the analog level around such that it crosses neighboring digitization levels. Therefore, as you average over a set of digitized values you actually get something which is close to the true analog level. Let's see this via example.

Suppose the noise we add is Gaussian distributed with $\sigma=2$. Then the distribution of the analog signal is

$$p(x) \propto \exp \left[ \frac{-(x - s(t))^2}{2\sigma^2} \right] = \exp \left[ \frac{-(x-0.8)^2}{8} \right] \, .$$

We now compute the average digital value $\langle s_n \rangle$ by averaging $p(x)$ over the integers. The normalization constant for the distribution is $$N = \sum_{m=-100}^{100} \exp \left[ \frac{-(m-0.8)^2}{8} \right]$$ and so we have $$\langle s_n \rangle = \frac{1}{N} \sum_{m=-100}^{100} m \exp\left[ \frac{-(m-0.8)^2}{8}\right] = 0.79999\ldots$$ Thus you can see that adding the white noise caused the average digitized signal to more closely match the true analog value.

Of course, adding the noise makes your signal to noise ratio worse. That means to to actually have a high probability of measuring the $\langle s_n\rangle$ we just computed, you have to take more samples than you would think in the noiseless case. This is why you hear about over-sampling and dithering at the same time. Because the dithering noise is truly uncorrelated, taking more samples always helps improve the signal to noise ratio, even if you're sampling way above the bandwidth of the analog signal coming in.

Dithering (intentionally adding noise) the input signal combined with oversampling can (no guarantees!) improve the effective number of bits in the signal and increase the signal to noise ratio.

Assuming the underlying process is a white noise process, the digitized signal will still look like white noise if the quantization level is well into the noise. This is in general a good thing. White noise has rather nice mathematical properties. Dealing with the non-linear, signal-dependent quantization noise is difficult. That quantization noise can generally be ignored if the quantization level is well into the noise.

What if the quantization level isn't well into the noise? Now you have messy quantization noise to deal with. One way to deal with it is to use a higher resolution ADC and make the quantization be well into noise. An alternative is to intentionally add white noise to the input signal so as to make the quantization noise from your cheap ADC small compared to the (dithered) signal noise.

The basic idea is to dither the input signal so that the noise in the dithered signal dominates over quantization noise (but obviously not by too much). The dithered signal is intentionally oversampled and then downsampled by averaging two or more successive measurements. This averaging only buys something if the input signal is noisy compared to the quantization noise. Averaged quantization noise looks like quantization noise. Averaging white noise lets signal rise out of the noise.

For more, read Walt Kester's tutorial The Good, the Bad, and the Ugly Aspects of ADC Input Noise—Is No Noise Good Noise?.

• @ThePhoton: Ack! That's was a typo. He asked for the spectral density of quantization noise. Ugh, and now I can't edit the comment. – DanielSank Dec 29 '14 at 16:53
• Reposting comment with fix: The fact that quantization noise is not additive got me into trouble in a job interview. The guy asked "what's the spectral density of quantization noise"? I explained that it doesn't really have a spectral density and tried to show him why, but he didn't get. I think he just wanted the canned answer for the case where you assume the digital levels are already into the analog noise (as mentioned in your answer) in which case the digitization noise is white, as you say. – DanielSank Dec 29 '14 at 16:54

Yes, adding uncorrelated noise to a signal, then averaging multiple samples, is one way of dealing with quantization noise.

To make this obvious, think of the limiting case. You want to measure the value of a signal between 0 and 1, but all you have is a 1-bit A/D. The A/D output is 0 when the signal is from 0 to .5, and 1 when the signal is .5 to 1.

Now consider a signal that is .3. With a nice clean .3 in, the A/D will always yield 0. You can't distinguish between a input signal of .1, .25, .3, etc, for example.

Now add random noise of ±.5. The signal seen by the A/D will now be somewhere from -.2 to .8. A 0 output has a probability of .7, and a 1 output of .3. Any one reading won't tell you much (unlike before when it told you the signal was from 0 to .5), but after a bunch of readings you get a reasonable idea of the signal. For example, if you take 100 readings, around 30 of them will be 1 and 70 will be 0. There is no guarantee of this, but the more readings you average, the higher the confidence that the average result represents the signal.

So dithering is a means of trading off certainty and bandwidth to get resolution. Note that no matter how many readings you take, you don't know the signal is within some bound, only that the probability of it being within some bound is high. Also note that you have lost bandwidth since many samples are required to get one signal value. Nonetheless, in many cases these are useful tradeoffs.

The above answers, dithering, are right but I would like to explicitly point out that you are explicitly trading resolution for time. A time vs. accuracy tradeoff. The most dramatic case is delta-sigma converters. They have (well there are probably exceptions) 1 bit of accuracy but by overclocking the conversion they are able to extract 24 bits, or so, of resolution. In this case they get around the tradeoff by overclocking (a lot).

Of course many other techniques are used in real converters; but the principle applies.

A long time ago (late 80's), we needed the ability to create an energy spectrum of incoming radiation as part of a gamma camera which is used in Nuclear Medicine. This required a fast ADC with excellent differential linearity (basically, every "bin" of the ADC has to be exactly the same width or the analog signal, which is continuously distributed, will give you a "bump" in a bin that is a little bit too wide - and this might look like a spectral peak).

The solution was an interesting variant on dithering: we had a small DAC and a counter, and used this to add a known amount of signal to the input. The sum signal was then digitized using a fast ADC with (comparatively) poor differential linearity, and the value of the DAC was subtracted again after the conversion; after this, the counter (input to the DAC) was incremented by 1 (with overflow resetting it back to zero). The result of this was that a voltage of the same value would be converted in one of many bins - in effect reducing the differential nonlinearity of the ADC by the size of the DAC counter.

You could do the same thing with random noise - but if you don't know what noise you are adding, you are of course degrading your signal. When the noise is roughly of the size of a single bin in the ADC, and you can take multiple measurements, this will help you estimate a signal to better than the resolution of the ADC - assuming that you can trust the differential linearity. But adding a known "random" signal (although we used a counter, it was "random in time" what value would happen to be present for a particular conversion) and taking account of its value allows you even greater precision.