On one hand, there is the statistical definition:

$R_x(\tau) = \frac{\operatorname{E}[(X_t - \mu)(X_{t+\tau} - \mu)]}{\sigma^2}$

and on the other hand, there is this other definition (e.g. see Dilip's comment here):

$R_x (\tau) = \int_{-\infty}^{\infty} x(t) x(t+\tau) {\rm d}t$

Why are these two definitions different? Is the first one defined for stochastic processes, while the second one defined for deterministic signals?

If so, does the Wiener-Khinichin theorem hold for the deterministic variant? (the Wikipedia article only discusses it in the context of a stochastic process)


You are correct that these two definitions are for different processes.

$R_x\left(\tau\right) = \frac{1}{\sigma^2} \mathbb{E}\left[\left(X_t - \mu\right)\left(X_{t+\tau} - \mu\right)\right]$

is defined for a stochastic process $X$ distributed according to a law $\nu$.

Whilst for a deterministic, finite-energy signal, $x\left(t\right)$ the autocorrelation is:

$R_x\left(\tau\right) = \int_{-\infty}^\infty x\left(\tau+t\right)x\left(\tau\right)dt$

The Wiener-Khinchin theorem will hold for any square-integrable deterministic $x\left(t\right)$ that is:

$\int_{-\infty}^\infty |x\left(\omega\right)|^2 d\omega < \infty$

so that the Fouier integral exists and converges in the $L^2$ sense (i.e. signals with finite energy).

The two definitions are different because the integrals used in the definitions are different. The deterministic case implicitly assumes a Riemann integral - that the process is nice in some way - it is continuous, and if differentiable the derivatives are bounded etc. However for random processes, $dt$ is difficult to define - the process could jump from $-\infty$ to $\infty$ over any inteval I could possibly define, so the Riemann limit isn't appropriate.

Instead we use the Lebesgue integral, where we can define the expectation of a random process: all that's required is that the range is smooth (for example $\mathbb{R}$), and we can still take the expectation with respect to some measure, modulo some restrictions on the random process (so that the integral converges in a monotone or dominated sense).

Integration of Random Process is a tricky business, and the definitions are written differently to keep people mindful of what they are working with.

One other note - the deterministic case isn't so interesting, as we can take the FT of a deterministic signal: the whole point of the WK theorem is for random processes - where the Fourier integral may not exist. I find it a particularly satisfying theorem, and the work to understand it definitely worthwhile.

  • $\begingroup$ Is there an equivalent definition for the autocorrelation for finite power signals, not the finite energy one you gave in your answer? $\endgroup$
    – teeeeee
    Apr 2 '20 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.