Rigorous derivation of autocorrelation of white noise

Question

It is said that the autocorrelation of white noise is the dirac delta function $\delta(\tau)$, but I don't know how to derive that...

Since white noise is a function with constant power spectral density (psd), i.e. $S(\omega) = C$, and by the Wiener-Kinchin theorem its autocorrelation function is the inverse Fourier transform of the psd, then we have $$ \begin{align*} G(\tau) &= \int_{-\infty}^{\infty} e^{i\omega \tau} S(\omega) d\omega \\ &= \int_{-\infty}^{\infty} e^{i\omega \tau} C d\omega \end{align*} $$ Then clearly $G(0) = \int_{-\infty}^{\infty} Cd \omega$ which diverges. Additionally, if $\tau \neq 0$ then $G(\tau) = \int_{-\infty}^{\infty} e^{i\omega \tau}Cd\omega$ which does not converge $\forall \tau \neq 0$. So it looks like the autocorrelation of a white noise function is not well defined. Why then is it said that it is $\delta(\tau)$?

Answer 1

Talking about this in the continuous world is a bit of a challenge because of the Dirac delta function. It's far easier to exemplify this in the discrete world with a Kronecker delta (which actually came before the Dirac delta), and then attempt to extend it to the continuous case.

The IDFT is defined as \begin{equation} x[n] = \sum_{k=0}^{N-1}X[k]e^{j2\pi\frac{kn}{N}} \end{equation}

If we let $X[k] = c$, we can say \begin{equation} x[n] = c\sum_{k=0}^{N-1}e^{j2\pi\frac{kn}{N}} \end{equation}

Since each of the DFT basis functions are periodic within the DFT window, and the integral of any oscillatory function with zero mean over exactly one period is zero, it is readily verified that \begin{equation} x[n] = \begin{cases} cN & n = 0 \newline 0 & else \end{cases} \end{equation}

So, the proof that, for the discrete case, the autocorrelation function of white noise is a Kronecker delta is pretty trivial. However, it is not so trivial for the continuous case. As you point out, the integral of a constant diverges. However, you seem to assume that the definition of the delta function is \begin{equation} \delta(t) = \begin{cases} \infty & t=0 \newline 0 & else \end{cases} \end{equation} but this is not the case. The actual definition of the delta function is \begin{equation} \int_{-\infty}^{\infty}f(x)\delta(x)dx = f(0) \end{equation} That is to say, the delta function integrates to one and effectively samples the function being multiplied by the Dirac delta.

If we have \begin{equation} G(\tau) = \int_{-\infty}^{\infty}S(\omega)e^{j\omega\tau}d\omega \end{equation} it is true that $G(\tau)$ diverges $\forall \, \tau$. However, at $\tau = 0$, $G(\tau)$ blows up to $\infty$, but for $\tau \neq 0$, the integral oscillates about zero. The idea that is that we assume that an infinite length function with finite period is periodic within an infinite length window, and therefore integrates to zero if this function has zero mean. So, you achieve the second definition of the Dirac delta function when you integrate over an infinite period.

This is a pretty informal treatment of the Dirac delta function, but one suitable for the signal processing stack exchange. Technically, the delta function is a distribution. A proof using this more rigorous notation is given by this answer on the math stack exchange.

EDIT Response to comments

The best way to recognize, in my opinion, that the delta function integrates to one is to view it from the perspective of a standard normal distribution. The standard normal distribution is defined as \begin{equation} \mathcal{N}(\mu,\sigma^{2}) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}} \end{equation}

If we allow the mean to be 0 and take the limit of the distribution, we find that \begin{equation} \delta(x) = \lim_{\sigma^{2}\rightarrow 0} \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{x^{2}}{2\sigma^{2}}} \end{equation}

We know that this integrates to one because all probability density functions integrate to 1.

Answer 2

This question is basically about the Fourier transform of the Dirac delta impulse and its inverse transform. From

$$\int_{-\infty}^{\infty}f(t)\delta(t)dt=f(0)\tag{1}$$

(assuming that $f(t)$ is continuous at $t=0$), we obtain for the Fourier transform of $\delta(t)$

$$\mathcal{F}\big\{\delta(t)\big\}=\int_{-\infty}^{\infty}\delta(t)e^{-j\omega t}dt=1\tag{2}$$

From the inversion formula we get the following integral

$$\frac{1}{2\pi}\int_{-\infty}^{\infty}e^{j\omega t}d\omega=\frac{1}{2\pi}\int_{-\infty}^{\infty}\cos(\omega t)d\omega\stackrel{?}{=}\delta(t)\tag{3}$$

The integrals in $(3)$ are of course meaningless if interpreted in the conventional sense. However, they become meaningful if interpreted as distributions.

In the following I will try to make $(3)$ plausible. Mathematical rigour can be found on math.SE.

If you're familiar with signal processing theory you know that

$$h(t)=\frac{\sin(\omega t)}{\pi t}\tag{4}$$

is the impulse response of an ideal lowpass filter with cut-off frequency $\omega$. If we increase the cut-off frequency, more and more components of the input signal are passed without any distortion. In the limit, all input signals appear at the output without any degradation, so we should accept

$$\lim_{\omega\to\infty}\frac{\sin(\omega t)}{\pi t}=\delta(t)\tag{5}$$

With $(5)$ we can solve the integral in $(3)$:

\begin{align*} \frac{1}{2\pi}\int_{-\infty}^{\infty}\cos(\omega t)d\omega &=\lim_{\Omega\to\infty}\frac{1}{2\pi}\int_{-\Omega}^{\Omega}\cos(\omega t)d\omega \\&=\lim_{\Omega\to\infty}\frac{1}{2\pi}\frac{2\sin(\Omega t)}{t}\\&=\lim_{\Omega\to\infty}\frac{\sin(\Omega t)}{\pi t}\\&=\delta(t)\tag{6} \end{align*}

Answer 3

Uhm, here is one of Matt's answers (maybe the only one) that I am simply dissatisfied with.

"I think you should be more explicit here in step 3."

---

Showing this Fourier pair:

$$ \mathscr{F} \Big\{ \delta(t) \Big\} = 1 $$

doesn't really do it. So that's fact, so what?

Again, the problem is more abstracted because there is no such thing as white noise. That is because the integral of the power spectrum is the power and the integral of a constant, non-zero function over all the reals is not a number. White noise has infinite power. So it's not even a finite-power signal (let alone a finite-energy signal). You can only approach the notion of white noise as a limit. That's the only way we, in the physical sciences, can deal with the notion of infinity.

We discussed this a little in the past.