Autocorrelation function and correlation integral

Question

I am confused by the definition of autocorrelation function. It is originally defined as the expected value $$R_{XX}(\tau) = E[(X(t)X(t+\tau)] = \langle X(t)X(t+\tau)\rangle\tag{1}$$ where $\langle\cdot\rangle$ is the ensemble average, and suppose they are all real signals. However, I see another definition defined by the correlation integral (e.g., see Autocorrelation - Wiki), which reads $$R_{XX}(\tau) = \int_{-\infty}^\infty X(t)X(t+\tau)dt \tag{2}$$ Under assumption of ergodicity, Eq.(1) can be approximated by temporal averaging, i.e., $$R_{XX}(\tau) = \lim_{T\to\infty}\frac{1}{2T}\int_{-T}^TX(t)X(t+\tau)dt\tag{3}$$ But I don't see how Eq.(3) turning into Eq.(2), or how Eq.(2) being equivalent to Eq.(1).

So am I misunderstanding or missing anything here, or the one defined by correlation integral is another version of "autocorrelation"?

Any hints would be helpful!

Thanks!

EDIT: Another question is, for Eq.(2), if one sets $\tau=0$, the result is expected to be the variance (or "energy") of the signal. However, if we start from Eq.(2), $$R_{XX}(0) = \lim_{T\to\infty}\frac{1}{2T}\int_{-T}^T X^2(t)dt$$ For any signal in $\mathcal{L}^2$ space (which has finite energy), the integral will be finite, but as $T\to\infty$, $R_{XX}(0)\to0$; that's not physical. Where did I go wrong?

Answer 1

The definition of auto-correlation depends on the type of signal. For random processes, the auto-correlation function is defined by the expectation given in Eq. $(1)$ of your question.

For deterministic signals, there are two definitions, depending on whether the signal is an energy signal (i.e., has finite energy), or a power signal (i.e., has finite power but infinite energy). In the first case, the auto-correlation is defined by Eq. $(2)$ in your question. In the latter case, the integral $(2)$ doesn't exist, and the auto-correlation is defined by the limit given in Eq. $(3)$.

Answer 2

Equation (3) is a practical symmetrical estimation of Equation (2) when the observation time $T$ increases. Equation (2) is a reformulation of Equation (1) using the concept of ergodicity.

Here, $X(t)$ is modeled as a continuous random variable. I see it as a variable phenomenon that can takes observed values at time $t$, depending on some probability law $p_X(u)$. Those values can in generally be "anything", and we cannot say more about that.

In analyzing such a process, you can be interested in some "expected values" that one is more likely to consider. For instance, you could be interested in the mean of $X$ for each time $t$. This is the "meaning" of what expectation does:

$$\mu_X(t) = E[X(t)] $$

This quantity is evaluated for each $t$ by integrating the probably density function of $X$. Many other moments of $X$ can be evaluated in the same way, so you can replace $X$ in the above formula by $X(t)X(t+\tau)$, and you get the process autocorrelation, a kind on average of the product at lag $\tau$.

Very often, the actual probably density function is not known, and maybe we don't need it to access to those average values. So we often make additional assumptions, like stationnarity (the probably does not change over time) and ergodicity (the expected values of "some property" at time $t$ can be estimated by averaging over times. This is typically done with discrete signals, as answered by Matt L.. You can learn more with the excellent SE.DSP answer of Dilip Sarwate What is the distinction between ergodic and stationary?.

So with an ergodicity assumption, one can estimate moments (eg autocorrelation) by time averaging (Eq. 2). Alas, we generally don't know the process over all times. Most signals have finite support. So, one expects that the knowledge would be better as $T$ increases. And a part of process analysis and spectral estimation is devoted to "how fast we converge to the true expectation" as $T\to\infty$, in order to bound estimation errors.

As there are several avatars of ergocity, I will summarized the definitions given in Ergodic process: if $X(t)$ is a wide-sense stationary process, with constant zero-mean $\mu_X(t)=0$, with autocovariance:

$$R_{XX}(\tau) = E([X(t)X(t+\tau)])$$

which is called an ensemble average. Let us define the symmetric time average quantity:

$$r_{XX}(T) = \frac{1}{2T}\int^{-T}_{-T}X(t)X(t+\tau)\mathrm{d}t$$

Then, the process $X$ is said to be autocovariance-ergodic if $r_{XX}(T)$ converges in the least-squares sense to $R_{XX}(\tau)$ as $T\to \infty$.

For dynamical systems, or for processes with known generating mechanism or even better known probability law, ergodicity can be proved or disproved... But this is more a definition/hypothesis than something that could be mathematically derived in general (as far as I know).

Related answers:

• What is the distinction between ergodic and stationary?
• What is a good example of an ergodic process?
• What's the meaning of ergodicity?

Answer 3

I think

$$R_{xx}(\tau) \triangleq \int_{-\infty}^\infty x(t)x(t+\tau)dt$$

Is just for a continuous signal $X$ that isn't a stochastic process. When it becomes a stochastic process, the expected value arrives to compute the ensemble average.

In this case, that is the definition of a finite duration real signal (which is obviously aperiodic) (and is the mean autocorrelation over the symbol times the time period)

This can be rewritten as:

$$R_{xx}(\tau) \triangleq \int_{t_0}^{t_0+T}x(t)x(t+\tau) \,dt$$

For an infinite duration real periodic signal it is (the mean autocorrelation within a period):

$$R_{xx}(\tau) \triangleq \lim_{T\to\infty} \frac{1}{2T} \int_{-T}^T x(t)x(t+\tau)dt $$

Which can be rewritten as:

$$R_{xx}(\tau) \triangleq \left(\frac{1}{T}\right)\int_{t_0}^{t_0+T} x(t)x(t+\tau) \,dt$$

The brackets denoting that it can also be derived from both the aperiodic definition where one symbol contains one period.

For an infinite aperiodic signal i.e. $|t|$, the autocorrelation doesn't exist.

For wide-sense-stationary complex random process $X$, the autocorrelation is defined as

$$R_{xx}(\tau) \triangleq \operatorname{E}\left[x(t)\overline{x(t-\tau)}\right]$$

This is the correlation coefficient of the two random variables without normalisation, so without subtracting the product of the means and dividing by the standard deviations.

And if it is an ergodic process, this expected value can be replaced with a limit of the time average:

$$R_{xx}(\tau) \triangleq \lim_{T\to\infty} \frac{1}{2T}\int^{-T}_{-T}\overline{x(t)}x(t+\tau)\mathrm{d}t$$

When the lag extends outside of the range of integration, the result of $x(t+\tau)$ is assumed to be 0 for a finite duration aperiodic signal, but has a defined value for an infinite waveform