# explanation of correlation of stationary stochastic processes

I have some doubts about correlation in stationary stochastic processes.

I know that the expectation of a random variable is $$E(x)=\int_{-\infty}^{+\infty} a f_x(a)da$$ and the mutual correlation of two variables is $E(x_1 x_2)$.

In two stationary stochastic processes the mutual correlation should be $E(x_1(t+\tau)x_2(t))$ now when and why I can translate this formula in: $$\int_{-\infty}^{+\infty} x_1(t+\tau)x_2(\tau)d\tau$$ I have not understand where this last formula comes from.

## 1 Answer

You are mixing up two different notions.

Your random process is a collection of random variables $\{X(t)\colon -\infty < t < \infty\}$, one random variable for each time instant. The autocorrelation function of the process is $$R_X(t, \hat{t}) = E[X(t)X(\hat{t})], -\infty < t, \hat{t} < \infty$$ and is a two-dimensional function (meaning it has two arguments, and if you try to visualize it, you should imagine a surface above (and possibly below) the plane with coordinate axes $t$ and $\hat{t}$. Definitely above because $R_X(t,t) = E[(X(t))^2]\geq 0$ for all $t$ (and processes for which $R_X(t,t) = 0$ for all $t$ are kinda boring).

For a stationary process, $R(t,\hat{t})$ depends only on the difference $\hat{t}-t$ of the two arguments, and it is common practice to write it as a function of one variable $\tau = \hat{t}-t$. Thus, $$\mathcal R(\tau) = R(t,\hat{t}) = E[X(t)X(t+\tau)].$$

The other notion is that if the experiment is performed, then each random variable takes on a numerical value. Thus, $X(t)$ takes on a specific numerical value that we name as $x(t)$ while $X(\hat{t})$ takes on some other numerical value that we name as $x(\hat{t})$ and so on. Thus, we have a real-valued function (or signal) $x(t)$ (nothing random about that!) for which the autocorrelation function is defined as $$R_x(\tau) = \int_{-\infty}^\infty x(t)x(t+\tau)\,\mathrm dt.$$ Note that $x(t)$ is called a sample path or realization of the random process: it is what you observe on your oscilloscope if you lift up your head from the math a little bit and look at the real world.

It is every DSPer's fondest hope that $R_x(\tau)$ is the same function as $\mathcal R(\tau)$. Unfortunately, nothing in what has been said above justifies this hope. But, since people like to assume that it is indeed true that $R_x(\tau) = \mathcal R_X(\tau)$, mathematicians have come up with a notion called ergodicity which (for our purposes) says that for an ergodic process, the ensemble means (expectations) equal the corresponding time-averages. As a practical matter, all model-builders assume that the processes that they want to model are indeed both stationary and ergodic, and decide on what the mean and the autocorrelation function etc should be by making observations of actual sample paths etc.

For more than what you probably want to know about random processes, see this answer of mine, and for an example of a (boring) stationary random process that is not ergodic, see this other answer of mine.

• Thanks, you have really clarified my ideas. I would like to ask you in practice what do I need autocorrelation? I used mutual correlation to compare two signals, but what about autocorrelation? – Andrea Nov 18 '15 at 15:28
• The mutual correlation (usually called the cross-correlation) does indeed tell you the similarity between two signals. Computing $\int x(t)y(t+\tau) dt$ for various values of $\tau$ tells you for which relative delays between $x(t)$ and $y(t)$ the two signals are most similar, but the differences between most similar and least similar are relatively small. When $x(t)$ and $y(t)$ are the same signal with different delays, there is a very pronounced peak in the cross-correlation function at the value of $\tau$ which aligns the two signals perfectly. (continued) – Dilip Sarwate Nov 18 '15 at 15:57
• (continuation) In fact, this peak is used for estimating the delay between a radar return (weak, noisy reflection from a target of the transmitted signal) and the transmitted signal itself. Why the value of this delay might be of any interest to a radar engineer or the engineer's bosses or customers is something I will leave to you to decide. – Dilip Sarwate Nov 18 '15 at 16:01
• Yes, I know that the cross-correlation is often used to find the $\tau$ of the noise in the signal, but I wanted to ask which information can I get from the autocorrelation of the realization of a random process. – Andrea Nov 18 '15 at 16:22
• The autocorrelation function of the realization is a model for (or estimate of) the autocorrelation function of the random process. The information that you get from the model or estimate is the mean and variance of each of the random variables $X(t)$ (same for all the random variables) as well as the correlation between each pair of random variables $X(t)$ and $X(t+\tau)$. For Gaussian processes, this is sufficient information to write down the joint density of any $n$ random variables in the process. See my other answer that I referred you to for details. – Dilip Sarwate Nov 18 '15 at 16:32