# Autocorrelation function and correlation integral

I am confused by the definition of autocorrelation function. It is originally defined as the expected value $$R_{XX}(\tau) = E[(X(t)X(t+\tau)] = \langle X(t)X(t+\tau)\rangle\tag{1}$$ where $$\langle\cdot\rangle$$ is the ensemble average, and suppose they are all real signals. However, I see another definition defined by the correlation integral (e.g., see Autocorrelation - Wiki), which reads $$R_{XX}(\tau) = \int_{-\infty}^\infty X(t)X(t+\tau)dt \tag{2}$$ Under assumption of ergodicity, Eq.(1) can be approximated by temporal averaging, i.e., $$R_{XX}(\tau) = \lim_{T\to\infty}\frac{1}{2T}\int_{-T}^TX(t)X(t+\tau)dt\tag{3}$$ But I don't see how Eq.(3) turning into Eq.(2), or how Eq.(2) being equivalent to Eq.(1).

So am I misunderstanding or missing anything here, or the one defined by correlation integral is another version of "autocorrelation"?

Thanks!

EDIT: Another question is, for Eq.(2), if one sets $$\tau=0$$, the result is expected to be the variance (or "energy") of the signal. However, if we start from Eq.(2), $$R_{XX}(0) = \lim_{T\to\infty}\frac{1}{2T}\int_{-T}^T X^2(t)dt$$ For any signal in $$\mathcal{L}^2$$ space (which has finite energy), the integral will be finite, but as $$T\to\infty$$, $$R_{XX}(0)\to0$$; that's not physical. Where did I go wrong?

The definition of auto-correlation depends on the type of signal. For random processes, the auto-correlation function is defined by the expectation given in Eq. $$(1)$$ of your question.

For deterministic signals, there are two definitions, depending on whether the signal is an energy signal (i.e., has finite energy), or a power signal (i.e., has finite power but infinite energy). In the first case, the auto-correlation is defined by Eq. $$(2)$$ in your question. In the latter case, the integral $$(2)$$ doesn't exist, and the auto-correlation is defined by the limit given in Eq. $$(3)$$.

• Do you think, under some appropriate assumptions, Eq.(1) can be transformed into Eq.(2)? Because I don't see a way to do this. – SJ H May 24 at 22:36
• @sj-h: Eq. (1) cannot be transformed into Eq. (2), or vice versa. However, if you have an ergodic process, then the time average (Eq. (3)) converges to the mean over the state space or ensemble (Eq. (1)). – Joe Mack Jun 24 at 15:20

Equation (3) is a practical symmetrical estimation of Equation (2) when the observation time $$T$$ increases. Equation (2) is a reformulation of Equation (1) using the concept of ergodicity.

Here, $$X(t)$$ is modeled as a continuous random variable. I see it as a variable phenomenon that can takes observed values at time $$t$$, depending on some probability law $$p_X(u)$$. Those values can in generally be "anything", and we cannot say more about that.

In analyzing such a process, you can be interested in some "expected values" that one is more likely to consider. For instance, you could be interested in the mean of $$X$$ for each time $$t$$. This is the "meaning" of what expectation does:

$$\mu_X(t) = E[X(t)]$$

This quantity is evaluated for each $$t$$ by integrating the probably density function of $$X$$. Many other moments of $$X$$ can be evaluated in the same way, so you can replace $$X$$ in the above formula by $$X(t)X(t+\tau)$$, and you get the process autocorrelation, a kind on average of the product at lag $$\tau$$.

Very often, the actual probably density function is not known, and maybe we don't need it to access to those average values. So we often make additional assumptions, like stationnarity (the probably does not change over time) and ergodicity (the expected values of "some property" at time $$t$$ can be estimated by averaging over times. This is typically done with discrete signals, as answered by Matt L.. You can learn more with the excellent SE.DSP answer of Dilip Sarwate What is the distinction between ergodic and stationary?.

So with an ergodicity assumption, one can estimate moments (eg autocorrelation) by time averaging (Eq. 2). Alas, we generally don't know the process over all times. Most signals have finite support. So, one expects that the knowledge would be better as $$T$$ increases. And a part of process analysis and spectral estimation is devoted to "how fast we converge to the true expectation" as $$T\to\infty$$, in order to bound estimation errors.

As there are several avatars of ergocity, I will summarized the definitions given in Ergodic process: if $$X(t)$$ is a wide-sense stationary process, with constant zero-mean $$\mu_X(t)=0$$, with autocovariance:

$$R_{XX}(\tau) = E([X(t)X(t+\tau)])$$

which is called an ensemble average. Let us define the symmetric time average quantity:

$$r_{XX}(T) = \frac{1}{2T}\int^{-T}_{-T}X(t)X(t+\tau)\mathrm{d}t$$

Then, the process $$X$$ is said to be autocovariance-ergodic if $$r_{XX}(T)$$ converges in the least-squares sense to $$R_{XX}(\tau)$$ as $$T\to \infty$$.

For dynamical systems, or for processes with known generating mechanism or even better known probability law, ergodicity can be proved or disproved... But this is more a definition/hypothesis than something that could be mathematically derived in general (as far as I know).

• I have a hard time understanding how one can go from Eq(1) to Eq(2) under ergodic assumption. Do you happen to have any reference to recommend? – SJ H May 24 at 22:35
• There are far better description, yet could you look at the last link "what's the meaning of ergodicity"? – Laurent Duval May 24 at 22:43
• I went through both threads you mentioned, I think I know what ergodicity is. I'd just like to see some mathematical derivation showing how one can go from Eq(1) to Eq(2) (or vice versa). – SJ H May 25 at 2:20
• I don't know if this can be derived in general, and I don't think so. This is more a generic "definition" of a subclass of signals termed mean-ergodic or autocovariance-ergodic, which is heuristic in some cases. Of course, there are cases where this can be proven (with additional knowledge) or probed with ergodicity tests. I have updated the answer wrt to the above comment – Laurent Duval May 25 at 14:25
• So can I take it this way, that the Eq.(2) is another definition (may or may not be heuristic) of the autocorrelation? – SJ H May 25 at 22:47

I think

$$R_{xx}(\tau) \triangleq \int_{-\infty}^\infty x(t)x(t+\tau)dt$$

Is just for a continuous signal $$X$$ that isn't a stochastic process. When it becomes a stochastic process, the expected value arrives to compute the ensemble average.

In this case, that is the definition of a finite duration real signal (which is obviously aperiodic) (and is the mean autocorrelation over the symbol times the time period)

This can be rewritten as:

$$R_{xx}(\tau) \triangleq \int_{t_0}^{t_0+T}x(t)x(t+\tau) \,dt$$

For an infinite duration real periodic signal it is (the mean autocorrelation within a period):

$$R_{xx}(\tau) \triangleq \lim_{T\to\infty} \frac{1}{2T} \int_{-T}^T x(t)x(t+\tau)dt$$

Which can be rewritten as:

$$R_{xx}(\tau) \triangleq \left(\frac{1}{T}\right)\int_{t_0}^{t_0+T} x(t)x(t+\tau) \,dt$$

The brackets denoting that it can also be derived from both the aperiodic definition where one symbol contains one period.

For an infinite aperiodic signal i.e. $$|t|$$, the autocorrelation doesn't exist.

For wide-sense-stationary complex random process $$X$$, the autocorrelation is defined as

$$R_{xx}(\tau) \triangleq \operatorname{E}\left[x(t)\overline{x(t-\tau)}\right]$$

This is the correlation coefficient of the two random variables without normalisation, so without subtracting the product of the means and dividing by the standard deviations.

And if it is an ergodic process, this expected value can be replaced with a limit of the time average:

$$R_{xx}(\tau) \triangleq \lim_{T\to\infty} \frac{1}{2T}\int^{-T}_{-T}\overline{x(t)}x(t+\tau)\mathrm{d}t$$

When the lag extends outside of the range of integration, the result of $$x(t+\tau)$$ is assumed to be 0 for a finite duration aperiodic signal, but has a defined value for an infinite waveform