Why is white noise uncorrelated for any two different samples?

Consider white noise with spectrum density $$N_0/2$$, it is known that the autocorrelation is given by:

$$R(\tau) = \frac{N_0}{2}\delta(\tau)$$

Meanwhile the definition of delta function in the following is very informal: \begin{align} \delta(\tau) = \begin{cases} 0, \tau\neq 0\\ \infty, \tau=0 \end{cases} \end{align} The strict definition of delta function is a measure satisfying the following $$\int \delta(\tau) f(\tau)d\tau=f(0) \tag{1}$$

My question: since the delta function just satisfies $$(1)$$, how can we obtain the conclusion that $$x(t)$$ is uncorrelated with $$x(t+\tau)$$ for any $$\tau\neq 0$$? The key is that we do not have $$\delta(\tau)=0$$.

• What do you mean by "we do not have $\delta(\tau) = 0$"? May 14 at 2:04
• You're right that the formal definition does not strictly imply $\delta(t\neq0)=0$. But it does imply that it is zero "almost everywhere", which is short for saying that the set $\{t:\delta(t)\neq0\}$ is of measure $0$. That is somewhat weaker than your original statement but it's without practical consequence. May 14 at 11:26
• One thing that should be pointed out is that theoretical white noise is not really a thing because it requires infinite power. But we can discuss bandlimited white noise that is flat up to Nyquist and that has finite power. And you can sample it which then should be equivalent with what might come out of a good random number generator. May 14 at 19:09
• Please take a look at this answer. May 14 at 19:19

2 Answers

You are correct in pointing out that the definition of the Dirac delta is not that it equals zero at $$\tau \neq 0$$ and infinity at $$\tau = 0$$, but rather that it integrates to one over its entire range and effectively "samples" the function it is multiplied with at $$\tau = 0$$ when integrated, giving the notation $$$$\int\delta(\tau)f(\tau)d\tau=f(0)$$$$

Another notation that I find helpful is to view the delta function from a centered normal distribution $$$$\delta(x) \approx \lim_{\sigma\rightarrow 0} \frac{1}{\sqrt{2\pi\sigma^{2}}}e^{-\frac{x^{2}}{2\sigma^{2}}}$$$$ This approximation works since the integral of any PDF is $$$$\int_{-\infty}^{\infty}f(x)dx = 1$$$$

Practically speaking, in terms of proving that $$x(t)$$ is uncorrelated with $$x(t+\tau)$$, if you integrate $$r_{x}(\tau)$$ with any other function $$f(\tau)$$, you would get $$$$\int r_{x}(\tau)f(\tau)d\tau = \int \frac{N_{0}}{2}\delta(\tau)f(\tau)d\tau = \frac{N_{0}}{2}f(0)$$$$ thereby showing that $$r_{x}(\tau)$$ contributes a non-zero value only at $$\tau = 0$$, which shows that $$x(t)$$ is only correlated with a non-shifted version of itself.

If you want to delve further into the Dirac delta, you would have to delve into measure theory. However, further exploration into this topic is better asked on the Math SE, and I would not be qualified to give you that answer.

Well, the usefulness in this notation lies in the fact that you can use it to show that when multiplied-then-integrated with another function, you get that other function.

To see that, try this:

1. Assume an energy-bound function $$g(\tau)$$. (If it helps imagination, this is the time-reversed complex conjugate channel impulse response $$h^*(-\tau)$$; so, instead of convolving with $$h(\tau)$$ to model what happens to our noise as it goes through a system, we just correlate with $$g(\tau)$$. Same operation, different notation.)
2. correlate your (random) white noise $$X(t)$$ with that to get a (random) signal $$Y(t)$$
3. Write down the cross-corr $$R_{XY}$$; be sure to use different integration variables, so that you can see that you can "nest" these integrals
4. (optional but very helpful:) Close your eyes and say "Fubini is my friend, fear is the mind-killer, fear is the little death", as you
5. swap the order of the integrals: you get something like $$R_{XY}=\int R_{XX}(u)\cdot g(u) \,\mathrm du.$$
6. realize that if that autocorrelation $$R_{XX}$$ really has the properties of the Dirac delta, then you just invented a system identification method (a channel sounder, in our example from 1.): excite a linear, memory-having system with white noise and correlate its output with the same white noise; on expectation (and assuming ergodicity, thus after arbitrarily long sample correlation) you get the system impulse response out.
• It is not clear how this answer is at all relevant to the question asked viz. How can we deduce that $x(t)$ and $x(t+\tau)$ are uncorrelated if we assume that $x$ is a white noise process. May 13 at 15:39