# Does the auto-correlation function of stationary random process always converge?

The auto-correlation function of the stationary random process only depends on the time difference $\tau$.

http://web.ntpu.edu.tw/~yshan/chapter6_han.pdf

64th slide of this lecture note mentions that for a zero-mean process, the autocorrelation converges to zero as $\tau$ goes to infinity.

I want to know why.

• Hi: I think it's being stated as an assumption rather than as a property of a zero mean process. Certainly, there can be zero mean processes whose autocorr function does not converge to zero. Now, whether there are zero mean stationary processes, whose autocorrelation function does not converge to zero is a more interesting question whose answer is probably yes but I'm not sure about that. All stationary means is that the covariance of two observations at $t_{1}$ and $t_{2}$ is only a function of $t_{1} - t{_2}$ but whether the covariance converges to zero is I think a different issue. – mark leeds Sep 12 '18 at 9:08
• Take a look at this related question. – Matt L. Sep 12 '18 at 9:17

No it does not necessarily.

For example the following discrete-time, WSS random process

$$x[n] = A \sin(\omega_0 n + \phi)$$

which is called the random phase sinusoid, where $A$ and $\omega_0 \neq 0$ are fixed values and $\phi$ is a random variable uniformly distributed in $\phi \in [-\pi,\pi)$ has an auto-correlation function of the form

$$r_{xx}[k] = \frac{A^2}{2} \cos(\omega_0 k)$$

which does not go to zero as $k$ goes to infinity; $\lim_{k \to \infty} r_{xx}[k] \neq 0$.

Similarly for a continuous-time process, the same can be shown.

Note, however, that as MattL indicated in his answer as well, the information contained within a random process is mostly included in its innovations part, this is also expressed in Wold decomposition theorem that any random process can be broken into two parts as a predictable periodic part and a regular unpredictable part (which is the innovations part), then if a WSS random process only includes a regular part but no predictable part, then its covariance (or correlation if zero mean) sequence shall go to zero as the lag goes to infinity if it has a uniformly convergent power spectral density; i.e., the DTFT of auto-correlation sequence (ACS) should converge, and this requires the sum of abs value of ACS be finite and this also requires that $\lim_{k \to \infty} r_{xx}[k] = 0$ for a zero-mean, WSS process containing no predictable (periodic) parts in it.

Furthermore, when the concept of ergodicity is introduced, then one of the necessary conditions for a WSS random process to be ergodic in the mean is that its auto-covariance (or equivalently auto-correlation for a zero mean process) should go to zero as $k$ goes to infinity. May be that was stated in that document.

A non-rigorous but intuitive explanation would be to note that for zero-mean (wide-sense) stationary processes, the autocorrelation at lag $\tau$ is the correlation between two samples of the process at a temporal distance $\tau$. It seems natural that with certain regularity assumptions (intuitive: "sufficient randomness"), that correlation should decrease (on average) with increasing lag, and should finally converge to zero for $|\tau|\rightarrow\infty$.

An example of a process for which this is not the case was given in Fat32's answer, but such a process lacks what I hand-wavingly referred to as "sufficient randomness", because it can be parameterized by a finite number of random variables. Such a process is sometimes called singular. Singular processes have Dirac delta impulses (at $\omega\neq 0$) in their power spectrum. Note that for the given example, the limit of the auto-correlation function does not exist.

The power spectrum of a non-singular (regular) process can have only one Dirac delta impulse, and that must be at $\omega=0$, reflecting a non-zero mean. Since the auto-correlation is the inverse Fourier transform of the power spectrum, that delta impulse causes a constant term in the auto-correlation, and, consequently, for a non-zero mean WSS process, the auto-correlation function cannot converge to zero, but it converges to the square of the mean:

$$\lim_{|\tau|\to\infty}R_X(\tau)=\mu_X^2\tag{1}$$

However, if $\mu_X=0$, you generally have (for regular processes)

$$\lim_{|\tau|\to\infty}R_X(\tau)=0\tag{2}$$

As a side note, since the power spectrum is the Fourier transform of the auto-correlation, the power spectrum can only exist as a conventional function (without Dirac delta impulses) if $(2)$ is satisfied, because otherwise the Fourier integral does not converge in the conventional sense.