# Rigorous derivation of autocorrelation of white noise

It is said that the autocorrelation of white noise is the dirac delta function $$\delta(\tau)$$, but I don't know how to derive that...

Since white noise is a function with constant power spectral density (psd), i.e. $$S(\omega) = C$$, and by the Wiener-Kinchin theorem its autocorrelation function is the inverse Fourier transform of the psd, then we have \begin{align*} G(\tau) &= \int_{-\infty}^{\infty} e^{i\omega \tau} S(\omega) d\omega \\ &= \int_{-\infty}^{\infty} e^{i\omega \tau} C d\omega \end{align*} Then clearly $$G(0) = \int_{-\infty}^{\infty} Cd \omega$$ which diverges. Additionally, if $$\tau \neq 0$$ then $$G(\tau) = \int_{-\infty}^{\infty} e^{i\omega \tau}Cd\omega$$ which does not converge $$\forall \tau \neq 0$$. So it looks like the autocorrelation of a white noise function is not well defined. Why then is it said that it is $$\delta(\tau)$$?

• Have you tried to compute the power spectrum of a process with autocorrelation equal to $\delta(\tau)$? What do you get? Commented May 31 at 6:32
• I can see that the power spectrum of a process with autocorrelation equal to $\delta(\tau)$ is just a constant function, but my confusion is that the inverse Fourier transform is only well defined for a function that vanishes sufficiently quickly at $\pm\infty$. Commented May 31 at 21:20
• Mashe, welcome to DSP.SE. This has been said by me (but also at least one of my profs) that true white noise is not a thing. Not a real, physical thing. That is because this white noise has infinite power. Check this out and you can see what approaching white noise in a limit looks like. Commented May 31 at 22:46

Talking about this in the continuous world is a bit of a challenge because of the Dirac delta function. It's far easier to exemplify this in the discrete world with a Kronecker delta (which actually came before the Dirac delta), and then attempt to extend it to the continuous case.

The IDFT is defined as $$$$x[n] = \sum_{k=0}^{N-1}X[k]e^{j2\pi\frac{kn}{N}}$$$$

If we let $$X[k] = c$$, we can say $$$$x[n] = c\sum_{k=0}^{N-1}e^{j2\pi\frac{kn}{N}}$$$$

Since each of the DFT basis functions are periodic within the DFT window, and the integral of any oscillatory function with zero mean over exactly one period is zero, it is readily verified that $$$$x[n] = \begin{cases} cN & n = 0 \newline 0 & else \end{cases}$$$$

So, the proof that, for the discrete case, the autocorrelation function of white noise is a Kronecker delta is pretty trivial. However, it is not so trivial for the continuous case. As you point out, the integral of a constant diverges. However, you seem to assume that the definition of the delta function is $$$$\delta(t) = \begin{cases} \infty & t=0 \newline 0 & else \end{cases}$$$$ but this is not the case. The actual definition of the delta function is $$$$\int_{-\infty}^{\infty}f(x)\delta(x)dx = f(0)$$$$ That is to say, the delta function integrates to one and effectively samples the function being multiplied by the Dirac delta.

If we have $$$$G(\tau) = \int_{-\infty}^{\infty}S(\omega)e^{j\omega\tau}d\omega$$$$ it is true that $$G(\tau)$$ diverges $$\forall \, \tau$$. However, at $$\tau = 0$$, $$G(\tau)$$ blows up to $$\infty$$, but for $$\tau \neq 0$$, the integral oscillates about zero. The idea that is that we assume that an infinite length function with finite period is periodic within an infinite length window, and therefore integrates to zero if this function has zero mean. So, you achieve the second definition of the Dirac delta function when you integrate over an infinite period.

This is a pretty informal treatment of the Dirac delta function, but one suitable for the signal processing stack exchange. Technically, the delta function is a distribution. A proof using this more rigorous notation is given by this answer on the math stack exchange.

The best way to recognize, in my opinion, that the delta function integrates to one is to view it from the perspective of a standard normal distribution. The standard normal distribution is defined as $$$$\mathcal{N}(\mu,\sigma^{2}) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}$$$$

If we allow the mean to be 0 and take the limit of the distribution, we find that $$$$\delta(x) = \lim_{\sigma^{2}\rightarrow 0} \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{x^{2}}{2\sigma^{2}}}$$$$

We know that this integrates to one because all probability density functions integrate to 1.

• Thank you the derivation of the autocorrelation in the discrete case is very helpful. I can believe that as the sampling rate approaches infinity, then the autocorrelation should converge to a dirac delta distribution. Commented May 31 at 21:22
• To clarify, I understand that $\delta(t) = \begin{cases} \infty & t=0 \newline 0 & else \end{cases}$ is not a rigorous definition of the dirac delta function, but in your last paragraph you show that $G(0)$ blows up to $\infty$ whereas $G(\tau) = 0$ for $\tau \neq 0$. How is this related to the actual definition of the delta function $\int_{-\infty}^{\infty} f(x)\delta(x)dx = f(0)$? Commented May 31 at 21:25
• @MasheBurnedead it's related because it shows that the delta function contributes a non-zero value to the integral only at $x=0$ and integrates to 1. Maybe I'm not quite following what you're getting at? Commented May 31 at 22:03
• Ah I see, I think I am almost on the same page now. So you showed that $G(\tau) = 0$ when $\tau \neq 0$, and $G(\tau)$ blows up when $\tau = 0$. If I understand correctly, in order to show that $G$ is the dirac delta function, we need to show that $G$ integrates to $1$, that is $\int_{-\infty}^{\infty}G(\tau)d\tau = 1$. How do we know this? Commented May 31 at 22:58
• @MasheBurnedead see the edit on my post Commented Jun 1 at 0:19

This question is basically about the Fourier transform of the Dirac delta impulse and its inverse transform. From

$$\int_{-\infty}^{\infty}f(t)\delta(t)dt=f(0)\tag{1}$$

(assuming that $$f(t)$$ is continuous at $$t=0$$), we obtain for the Fourier transform of $$\delta(t)$$

$$\mathcal{F}\big\{\delta(t)\big\}=\int_{-\infty}^{\infty}\delta(t)e^{-j\omega t}dt=1\tag{2}$$

From the inversion formula we get the following integral

$$\frac{1}{2\pi}\int_{-\infty}^{\infty}e^{j\omega t}d\omega=\frac{1}{2\pi}\int_{-\infty}^{\infty}\cos(\omega t)d\omega\stackrel{?}{=}\delta(t)\tag{3}$$

The integrals in $$(3)$$ are of course meaningless if interpreted in the conventional sense. However, they become meaningful if interpreted as distributions.

In the following I will try to make $$(3)$$ plausible. Mathematical rigour can be found on math.SE.

If you're familiar with signal processing theory you know that

$$h(t)=\frac{\sin(\omega t)}{\pi t}\tag{4}$$

is the impulse response of an ideal lowpass filter with cut-off frequency $$\omega$$. If we increase the cut-off frequency, more and more components of the input signal are passed without any distortion. In the limit, all input signals appear at the output without any degradation, so we should accept

$$\lim_{\omega\to\infty}\frac{\sin(\omega t)}{\pi t}=\delta(t)\tag{5}$$

With $$(5)$$ we can solve the integral in $$(3)$$:

\begin{align*} \frac{1}{2\pi}\int_{-\infty}^{\infty}\cos(\omega t)d\omega &=\lim_{\Omega\to\infty}\frac{1}{2\pi}\int_{-\Omega}^{\Omega}\cos(\omega t)d\omega \\&=\lim_{\Omega\to\infty}\frac{1}{2\pi}\frac{2\sin(\Omega t)}{t}\\&=\lim_{\Omega\to\infty}\frac{\sin(\Omega t)}{\pi t}\\&=\delta(t)\tag{6} \end{align*}

• Regarding $\frac{\sin(\omega t)}{\pi t} = \delta(t)$, I don't quite see why this is true. For a fixed $t \neq 0$, then $\frac{\sin(\omega t)}{t}$ doesn't converge for $\omega \rightarrow \infty$, since $\frac{\sin(\omega t)}{t}$ just oscillates around $0$ right? Commented Jun 1 at 23:43
• @MasheBurnedead: Note that this limit must be interpreted in a distributional sense, i.e., $$\lim_{\omega\to\infty}\int_{-\infty}^{\infty}\frac{\sin(\omega t)}{\pi t}f(t)dt=f(0)$$ for some function $f(t)$ that is continuous at $t=0$. This limit is a well-known result, see for instance Eq. (37) here. Commented Jun 2 at 11:36

Uhm, here is one of Matt's answers (maybe the only one) that I am simply dissatisfied with.

"I think you should be more explicit here in step 3."

Showing this Fourier pair:

$$\mathscr{F} \Big\{ \delta(t) \Big\} = 1$$

doesn't really do it. So that's fact, so what?

Again, the problem is more abstracted because there is no such thing as white noise. That is because the integral of the power spectrum is the power and the integral of a constant, non-zero function over all the reals is not a number. White noise has infinite power. So it's not even a finite-power signal (let alone a finite-energy signal). You can only approach the notion of white noise as a limit. That's the only way we, in the physical sciences, can deal with the notion of infinity.

We discussed this a little in the past.

• My argument wasn't about showing that the Fourier transform of the Dirac impulse is $1$. That's just a first step in order to introduce the inversion integral $\frac{1}{2\pi}\int_{-\infty}^{\infty}e^{j\omega t}d\omega$, which at first sight seems meaningless. What I hoped to contribute is to make intuitive sense of that integral in a way that is easy to understand for us engineers. Commented Jun 1 at 9:40