What is the distinction between ergodic and stationary?

Question

I have trouble distinguishing between these two concepts. This is my understanding so far.

A stationary process is a stochastic process whose statistical properties do not change with time. For a strict-sense stationary process, this means that its joint probability distribution is constant; for a wide-sense stationary process, this means that its 1st and 2nd moments are constant.

An ergodic process is one where its statistical properties, like variance, can be deduced from a sufficiently long sample. E.g., the sample mean converges to the true mean of the signal, if you average long enough.

Now, it seems to me that a signal would have to be stationary, in order to be ergodic. And what kinds of signals could be stationary, but not ergodic? If a signal has the same variance for all time, for example, how could the time-averaged variance not converge to the true value?

So, what is the real distinction between these two concepts? Can you give me an example of a process that is stationary without being ergodic, or ergodic without being stationary?

Answer 1

A random process is a collection of random variables, one for each time instant under consideration. Typically this may be continuous time ($-\infty < t < \infty$) or discrete time (all integers $n$, or all time instants $nT$ where $T$ is the sample interval). Stationarity refers to the distributions of the random variables, ergodicity takes into account the sample paths, i.e. what you observe physically. As you say, the average value or DC component of a sample path converges to the mean value of the process if the sample path is observed long enough, provided the process is ergodic and stationary, etc.

As an example of the difference between the two concepts, suppose that $X(t) = Y$ for all $t$ under consideration. Here $Y$ is a random variable. This is a stationary process: each $X(t)$ has the same distribution (the distribution of $Y$), same mean $E[X(t)] = E[Y]$, same variance etc.; each $X(t_1)$ and $X(t_2)$ have the same joint distribution (though it is degenerate) and so on. But the process is not ergodic because each sample path is a constant. Specifically, if a trial of the experiment (as performed by you, or by a superior being) results in $Y$ having value $\alpha$, then the sample path of the random process that corresponds to this experimental outcome has value $\alpha$ for all $t$, and the DC value of the sample path is $\alpha$, not $E[X(t)] = E[Y]$, no matter how long you observe the (rather boring) sample path. In a parallel universe, the trial would result in $Y = \beta$ and the sample path in that universe would have value $\beta$ for all $t$. It is not easy to write mathematical specifications to exclude such trivialities from the class of stationary processes, and so this remains as an example of a stationary random process that is not ergodic.

Answer 2

Let us consider a hypothetical random process where the sample functions are DC values and are different from each other:

X₁(t) = constant= mean of X₁(t)

X₂(t) = constant= mean of X₂(t)

The temporal mean of $X_1(t)$ and $X_2(t)$ are constant but not equal. if my process is stationary $X(t_1)$ and $X(t_2)$ are equal and RVs (refer Dilip's answer)

So ensemble mean of $X(t)$ is constant.

This ensemble mean is certainly not equal to the temporal mean of $X_1(t)$ and $X_2(t)$ (they themselves are not equal). This can be called a stationary but not an ergodic process.

In contrast , $X(t)=Acos(\omega t+\theta)$ where $\theta$ is an RV is ergodic.

Answer 3

An ergodic process is a process for which you can substitute the ergodic mean for the temporal mean.

The real mean, variance, etc... are defined by following a process over time and averaging, etc... For instance, if you want to know the mean of my size, you would have to average it from when I am born to when I die. Obviously the later example is not a stationary process.

The ergodic mean would be if instead of following my size over time, you would freeze time, and take the mean over a sample of different individual humans. There is no reason for these two means to be the same, so the process of my size is not ergodic.

That is a bad example, but it get more important if you consider the simple case of a gas at equilibrium. For instance the mean squared velocity is noted $\bar{V^2}$ (mean over time), but is often calculated by taking the ensemble mean $\left< V^2 \right>$: the mean of the square velocity of all molecules of the gas at an instant $t$.

Most thermodynamics theorems requires to use $\bar{V^2}$, but it is more easy to compute and use $\left< V^2 \right>$. The ergodic hypothesis is the hypothesis stating that it is right to substitute one for the other. An ergodic process is a process for which the ergodic hypothesis is true.

The ergodic hypothesis is false in the general case.