# What is the distinction between ergodic and stationary?

I have trouble distinguishing between these two concepts. This is my understanding so far.

A stationary process is a stochastic process whose statistical properties do not change with time. For a strict-sense stationary process, this means that its joint probability distribution is constant; for a wide-sense stationary process, this means that its 1st and 2nd moments are constant.

An ergodic process is one where its statistical properties, like variance, can be deduced from a sufficiently long sample. E.g., the sample mean converges to the true mean of the signal, if you average long enough.

Now, it seems to me that a signal would have to be stationary, in order to be ergodic.

• And what kinds of signals could be stationary, but not ergodic?
• If a signal has the same variance for all time, for example, how could the time-averaged variance not converge to the true value?
• So, what is the real distinction between these two concepts?
• Can you give me an example of a process that is stationary without being ergodic, or ergodic without being stationary?
-
You might want to look at this answer to a related question. – Dilip Sarwate Jan 17 '12 at 21:31
This lecture spells it literally that ergodic is a subset of stationary. I just cannot understand what Stationary Ergodic Process article is doing in Wikipedia? Does it mean that there is non-stationary ergodic process? – Val May 17 '14 at 16:32

A random process is a collection of random variables, one for each time instant under consideration. Typically this may be continuous time ($-\infty < t < \infty$) or discrete time (all integers $n$, or all time instants $nT$ where $T$ is the sample interval). Stationarity refers to the distributions of the random variables, ergodicity takes into account the sample paths, i.e. what you observe physically. As you say, the average value or DC component of a sample path converges to the mean value of the process if the sample path is observed long enough, provided the process is ergodic and stationary, etc.

As an example of the difference between the two concepts, suppose that $X(t) = Y$ for all $t$ under consideration. Here $Y$ is a random variable. This is a stationary process: each $X(t)$ has the same distribution (the distribution of $Y$), same mean $E[X(t)] = E[Y]$, same variance etc.; each $X(t_1)$ and $X(t_2)$ have the same joint distribution (though it is degenerate) and so on. But the process is not ergodic because each sample path is a constant. Specifically, if a trial of the experiment (as performed by you, or by a superior being) results in $Y$ having value $\alpha$, then the sample path of the random process that corresponds to this experimental outcome has value $\alpha$ for all $t$, and the DC value of the sample path is $\alpha$, not $E[X(t)] = E[Y]$, no matter how long you observe the (rather boring) sample path. In a parallel universe, the trial would result in $Y = \beta$ and the sample path in that universe would have value $\beta$ for all $t$. It is not easy to write mathematical specifications to exclude such trivialities from the class of stationary processes, and so this remains as an example of a stationary random process that is not ergodic.

-
I couldn't understand the example. If you say Y is a constant, then any path of x(t) is a constant. The mean of a constant is itself, hence E[X(t)] = E[Y] = Y. Unless I missed something. – Drazick Jan 18 '12 at 17:20
I added a few words to clarify the meaning. $Y$ is a random variable, not a constant. Its value on any trial of the experiment need not be the same as $E[Y]$. – Dilip Sarwate Jan 18 '12 at 17:40
Now I see your intention. I think it is important to note that usually the meaning of ergodic process is that by observing a single path in time one could deduce some properties of the RV as a whole. As you showed, in the above example, each path is a constant, yet the RV is a something with probability properties. Thanks for clarification. – Drazick Jan 18 '12 at 17:48
Thanks for the helpful answer. What do you think about a signal that is ergodic, but not stationary? Is that possible? – Matt Jan 23 '12 at 21:19
@Matt In the solution of the book "communication systems" simon haykin writes that "for a random process to be ergodic it has to be stationary" – Roney Island Aug 17 '12 at 4:46

Let us consider a hypothetical random process where the sample functions are DC values and are different from each other:

X1(t) = constant= mean of X1(t)

X2(t) = constant= mean of X2(t)

The temporal mean of $X_1(t)$ and $X_2(t)$ are constant but not equal. if my process is stationary $X(t_1)$ and $X(t_2)$ are equal and RVs (refer Dilip's answer)

So ensemble mean of $X(t)$ is constant.

This ensemble mean is certainly not equal to the temporal mean of $X_1(t)$ and $X_2(t)$ (they themselves are not equal). This can be called a stationary but not an ergodic process.

In contrast , $X(t)=Acos(\omega t+\theta)$ where $\theta$ is an RV is ergodic.

-

An ergodic process is a process for which you can substitute the ergodic mean for the temporal mean.

The real mean, variance, etc... are defined by following a process over time and averaging, etc... For instance, if you want to know the mean of my size, you would have to average it from when I am born to when I die. Obviously the later example is not a stationary process.

The ergodic mean would be if instead of following my size over time, you would freeze time, and take the mean over a sample of different individual humans. There is no reason for these two means to be the same, so the process of my size is not ergodic.

That is a bad example, but it get more important if you consider the simple case of a gas at equilibrium. For instance the mean squared velocity is noted $\bar{V^2}$ (mean over time), but is often calculated by taking the ensemble mean $\left< V^2 \right>$: the mean of the square velocity of all molecules of the gas at an instant $t$.

Most thermodynamics theorems requires to use $\bar{V^2}$, but it is more easy to compute and use $\left< V^2 \right>$. The ergodic hypothesis is the hypothesis stating that it is right to substitute one for the other. An ergodic process is a process for which the ergodic hypothesis is true.

The ergodic hypothesis is false in the general case.

-
I don't understand this answer. The process of Jolow's size is neither stationary nor ergodic whereas the OP was wondering if there can be a stationary process that is not ergodic. Is the answer essentially that in general the ergodic hypothesis is false and it is (sort of) universally true that the sample mean is different from the ensemble mean, just get used to it and live with it? – Dilip Sarwate Jan 17 '12 at 19:09
@DilipSarwate: after re-reading, it is a bad answer that does not answer the question, and I am considering deleting it. I was reminding my thermodynamics lectures, whereas the question was more about statistics... – Jean-Yves Jan 17 '12 at 20:00
@DilipSarwate what is Jolow's size? – Roney Island Aug 17 '12 at 5:01
@MichaelCorleone I do not remember what the reference to Jolow means. My guess is that Jean-Yves posted his answer under the nom-de-plume Jolow and I used that name in my answer, and that he has since then decided to use Jean-Yves as his user name on stackexchange. Such name changes are reflected on what is displayed on the screen but are not recorded as an edit of the answer. – Dilip Sarwate Aug 17 '12 at 13:29
@DilipSarwate: you are right indeed. Jolow is just my nickname. – Jean-Yves Aug 18 '12 at 6:28

as I underestand, example below shows a ergodic and stationary process

 X1 X2 X3  | mean var ...
1  2  3   | 2    1
2  3  1   | 2
3  1  2   | 2
----------


mean 2 2 2 var 1

because the mean and variance of every column is constant along the time and mean and variance of every row is constant along the time

-

I hope this video from 16:55 could clear your doubts