What is a good example of an ergodic process?

Question

I'm trying to find simple examples of an ergodic process. What process comes to your mind as a good illustration of its properties?

A quick research (Wikipedia, another answer) mainly gives examples of non-ergodic processes. Also, I'm wondering which real world phenomena lend itself to be modeled as an ergodic process?

Answer 1

Just suppose I give you a series of numbers, and I tell you they were picked randomly. And you know I am not trying to deceive you. Numbers are: $3$, $1$, $4$, $1$, $5$, $3$, $2$, $3$, $4$, $3$.

I now propose you to predict the next one, or at least, to be as close as possible. Which number would you pick?

[Think]

[Compute]

• I bet most of the readers are likely to choose a number between $0$ and $6$. Because of the limited span.
• Perhaps an integer. Who is likely to propose $\pi$ (even thinking of the first digits)?
• Possibly $2$, $3$, or $4$. Maybe even $3$.

Basically, you are assuming that I provided numbers with some unknown rule. And perhaps, you may think (or make the hypothesis) that the series of given numbers, if long enough, can provide you with a good understanding of the rules that I have in mind. If you do so, you hypothesize that my mental process is ergodic:

a process in which every sequence or sizable sample is equally representative of the whole (as in regard to a statistical parameter) (Merriam-Webster)

Here, there is no way to be sure that my series follows an ergodic process. 3432 is my card PIN, 3 a mistake (I intended 6, but I am clumsy), 4, 3, 1 and 5 are first digits of $\pi$ that I use quite often. My next "number" would have been C (in hexadecimal). I do not believe this process is ergodic. Each number comes out from different laws. But honestly, I do not know. Maybe I am subject to some higher order forces that drive me under ergodicity rules.

So, ergodicity is a hypothesis of a sort of "simplicity" in the rules of a process. Like stationarity or sparsity. Cast a regular die with $6$ faces. Toss a normal coin. If nothing outside tries to influence the result (an invisible being that catches the die and shows some face of its choice), you are likely to produce an ergodic process.

Instead of being able to toss an infinite number of coins, with your infinite number of thumbs, precisely at the same second, you toss one coin every second, and believe the final result is about the same.

The Brownian motion possesses ergodic properties too.

Answer 2

From the wikipedia article:

a stochastic process is said to be ergodic if its statistical properties can be deduced from a single, sufficiently long, random sample of the process.

In other words: the time-ensemble statistical properties are the same as the realization-ensemble statistical properties.

Maybe we need to take a step back and talk about what a stochastic process is, to get a start.

Imagine it is a stormy day. You sit at home, and look out the window. Occasionally, you see leaves being blown by your window. You get your whiteboard markers and draw a coordinate system onto your window, so you can now observe multiple leaf-paths and compare them:

So, each path is one realization of the "leafpaths on a stormy day" stochastic process.

Now, you start by just considering one of the paths: You call a friend, who's an excellent aerodynamics/physics guy, and the two of you derive a stochastical model for what's happening to objects getting blown around the house. It turns out, the $y$ position of a leave has a constant expectation value after averaging over $x$.

Now, you go and look at a fixed $x$-position, but average over all the hundreds of leaves. It turns out that, by careful mathematical modelling, the expectation of the $y$ value by averaging over all realizations takes the same value as when averaging a single realization over $x$.

Answer 3

It is usually more difficult to understand the non-ergodic case (that is why people look for examples of such processes more often).

As an example of ergodic process, let the process $X(t)$ represent repeated coin flips. At each time $t$, we have a random variable $X$ that can choose between $0$ or $1$. If it is a fair coin, then the ensemble mean is $\frac{1}{2}$ since the two possibilities are equiprobable.

Now if you repeat this trial for a large number of times like $N$ and then calculate the time average $m=\frac{X(1)+X(2)+\cdots}{N}$, then you can see that $m\to \frac{1}{2}$. Hence, the ensemble mean is equal to time average and the process is ergodic.

Regarding the second part of your question, we can use ergodicity to simplify problems. For example, between ensemble mean and the time average one might be difficult or even impossible to calculate (or simulate). But since we know (or assume) the process is ergodic (i.e they are identical), we just calculate the one that is simpler. As an example, I can think of Monte Carlo methods (such as those we use to model the error performance of a communication system) where we simulate the transmission-reception chain and repeat it for several times and average the results to find out about the ensemble properties (like the probability of error, etc.).