# Autocorrelation function and correlation integral

I am confused by the definition of autocorrelation function. It is originally defined as the expected value $$R_{XX}(\tau) = E[(X(t)X(t+\tau)] = \langle X(t)X(t+\tau)\rangle\tag{1}$$ where $$\langle\cdot\rangle$$ is the ensemble average, and suppose they are all real signals. However, I see another definition defined by the correlation integral (e.g., see Autocorrelation - Wiki), which reads $$R_{XX}(\tau) = \int_{-\infty}^\infty X(t)X(t+\tau)dt \tag{2}$$ Under assumption of ergodicity, Eq.(1) can be approximated by temporal averaging, i.e., $$R_{XX}(\tau) = \lim_{T\to\infty}\frac{1}{2T}\int_{-T}^TX(t)X(t+\tau)dt\tag{3}$$ But I don't see how Eq.(3) turning into Eq.(2), or how Eq.(2) being equivalent to Eq.(1).

So am I misunderstanding or missing anything here, or the one defined by correlation integral is another version of "autocorrelation"?

Thanks!

EDIT: Another question is, for Eq.(2), if one sets $$\tau=0$$, the result is expected to be the variance (or "energy") of the signal. However, if we start from Eq.(2), $$R_{XX}(0) = \lim_{T\to\infty}\frac{1}{2T}\int_{-T}^T X^2(t)dt$$ For any signal in $$\mathcal{L}^2$$ space (which has finite energy), the integral will be finite, but as $$T\to\infty$$, $$R_{XX}(0)\to0$$; that's not physical. Where did I go wrong?

The definition of auto-correlation depends on the type of signal. For random processes, the auto-correlation function is defined by the expectation given in Eq. $$(1)$$ of your question.

For deterministic signals, there are two definitions, depending on whether the signal is an energy signal (i.e., has finite energy), or a power signal (i.e., has finite power but infinite energy). In the first case, the auto-correlation is defined by Eq. $$(2)$$ in your question. In the latter case, the integral $$(2)$$ doesn't exist, and the auto-correlation is defined by the limit given in Eq. $$(3)$$.

• Do you think, under some appropriate assumptions, Eq.(1) can be transformed into Eq.(2)? Because I don't see a way to do this. – SJ H May 24 at 22:36
• @sj-h: Eq. (1) cannot be transformed into Eq. (2), or vice versa. However, if you have an ergodic process, then the time average (Eq. (3)) converges to the mean over the state space or ensemble (Eq. (1)). – Joe Mack Jun 24 at 15:20

Equation (3) is a practical symmetrical estimation of Equation (2) when the observation time $$T$$ increases. Equation (2) is a reformulation of Equation (1) using the concept of ergodicity.

Here, $$X(t)$$ is modeled as a continuous random variable. I see it as a variable phenomenon that can takes observed values at time $$t$$, depending on some probability law $$p_X(u)$$. Those values can in generally be "anything", and we cannot say more about that.

In analyzing such a process, you can be interested in some "expected values" that one is more likely to consider. For instance, you could be interested in the mean of $$X$$ for each time $$t$$. This is the "meaning" of what expectation does:

$$\mu_X(t) = E[X(t)]$$

This quantity is evaluated for each $$t$$ by integrating the probably density function of $$X$$. Many other moments of $$X$$ can be evaluated in the same way, so you can replace $$X$$ in the above formula by $$X(t)X(t+\tau)$$, and you get the process autocorrelation, a kind on average of the product at lag $$\tau$$.

Very often, the actual probably density function is not known, and maybe we don't need it to access to those average values. So we often make additional assumptions, like stationnarity (the probably does not change over time) and ergodicity (the expected values of "some property" at time $$t$$ can be estimated by averaging over times. This is typically done with discrete signals, as answered by Matt L.. You can learn more with the excellent SE.DSP answer of Dilip Sarwate What is the distinction between ergodic and stationary?.

So with an ergodicity assumption, one can estimate moments (eg autocorrelation) by time averaging (Eq. 2). Alas, we generally don't know the process over all times. Most signals have finite support. So, one expects that the knowledge would be better as $$T$$ increases. And a part of process analysis and spectral estimation is devoted to "how fast we converge to the true expectation" as $$T\to\infty$$, in order to bound estimation errors.

As there are several avatars of ergocity, I will summarized the definitions given in Ergodic process: if $$X(t)$$ is a wide-sense stationary process, with constant zero-mean $$\mu_X(t)=0$$, with autocovariance:

$$R_{XX}(\tau) = E([X(t)X(t+\tau)])$$

which is called an ensemble average. Let us define the symmetric time average quantity:

$$r_{XX}(T) = \frac{1}{2T}\int^{-T}_{-T}X(t)X(t+\tau)\mathrm{d}t$$

Then, the process $$X$$ is said to be autocovariance-ergodic if $$r_{XX}(T)$$ converges in the least-squares sense to $$R_{XX}(\tau)$$ as $$T\to \infty$$.

For dynamical systems, or for processes with known generating mechanism or even better known probability law, ergodicity can be proved or disproved... But this is more a definition/hypothesis than something that could be mathematically derived in general (as far as I know).