Signal Processing Stack Exchange is a question and answer site for practitioners of the art and science of signal, image and video processing. It only takes a minute to sign up.
Sign up to join this community
The auto-correlation function of the stationary random process only depends on the time difference $\tau$.
http://web.ntpu.edu.tw/~yshan/chapter6_han.pdf
64th slide of this lecture note mentions that for a zero-mean process, the autocorrelation converges to zero as $\tau$ goes to infinity.
I want to know why.
No it does not necessarily.
For example the following discrete-time, WSS random process
$$x[n] = A \sin(\omega_0 n + \phi) $$
which is called the random phase sinusoid, where $A$ and $\omega_0 \neq 0$ are fixed values and $\phi$ is a random variable uniformly distributed in $\phi \in [-\pi,\pi)$ has an auto-correlation function of the form
$$ r_{xx}[k] = \frac{A^2}{2} \cos(\omega_0 k) $$
which does not go to zero as $k$ goes to infinity; $ \lim_{k \to \infty} r_{xx}[k] \neq 0$.
Similarly for a continuous-time process, the same can be shown.
Note, however, that as MattL indicated in his answer as well, the information contained within a random process is mostly included in its innovations part, this is also expressed in Wold decomposition theorem that any random process can be broken into two parts as a predictable periodic part and a regular unpredictable part (which is the innovations part), then if a WSS random process only includes a regular part but no predictable part, then its covariance (or correlation if zero mean) sequence shall go to zero as the lag goes to infinity if it has a uniformly convergent power spectral density; i.e., the DTFT of auto-correlation sequence (ACS) should converge, and this requires the sum of abs value of ACS be finite and this also requires that $\lim_{k \to \infty} r_{xx}[k] = 0$ for a zero-mean, WSS process containing no predictable (periodic) parts in it.
Furthermore, when the concept of ergodicity is introduced, then one of the necessary conditions for a WSS random process to be ergodic in the mean is that its auto-covariance (or equivalently auto-correlation for a zero mean process) should go to zero as $k$ goes to infinity. May be that was stated in that document.
A non-rigorous but intuitive explanation would be to note that for zero-mean (wide-sense) stationary processes, the autocorrelation at lag $\tau$ is the correlation between two samples of the process at a temporal distance $\tau$. It seems natural that with certain regularity assumptions (intuitive: "sufficient randomness"), that correlation should decrease (on average) with increasing lag, and should finally converge to zero for $|\tau|\rightarrow\infty$.
An example of a process for which this is not the case was given in Fat32's answer, but such a process lacks what I hand-wavingly referred to as "sufficient randomness", because it can be parameterized by a finite number of random variables. Such a process is sometimes called singular. Singular processes have Dirac delta impulses (at $\omega\neq 0$) in their power spectrum. Note that for the given example, the limit of the auto-correlation function does not exist.
The power spectrum of a non-singular (regular) process can have only one Dirac delta impulse, and that must be at $\omega=0$, reflecting a non-zero mean. Since the auto-correlation is the inverse Fourier transform of the power spectrum, that delta impulse causes a constant term in the auto-correlation, and, consequently, for a non-zero mean WSS process, the auto-correlation function cannot converge to zero, but it converges to the square of the mean:
$$\lim_{|\tau|\to\infty}R_X(\tau)=\mu_X^2\tag{1}$$
However, if $\mu_X=0$, you generally have (for regular processes)
$$\lim_{|\tau|\to\infty}R_X(\tau)=0\tag{2}$$
As a side note, since the power spectrum is the Fourier transform of the auto-correlation, the power spectrum can only exist as a conventional function (without Dirac delta impulses) if $(2)$ is satisfied, because otherwise the Fourier integral does not converge in the conventional sense.
