11
$\begingroup$

I'm trying to figure out if there is a direct relationship between these concepts. Strictly from the definitions, they appear to be different concepts in general. The more I think about it, however, the more I think they are very similar.

Let $X,Y$ be WSS random vectors. The covariance, $C_{XY}$, is given by $$C_{XY}=E\left[(X-\mu_x)(Y-\mu_y)^H\right]$$ where $H$ stands for the Hermitian of the vector.

Let $Z$ be a WSS random vector. The autocorrelation function, $R_{XX}$, is given by $$R_{ZZ}(\tau)=E\left[\left(Z(n)-\mu_z\right)\left(Z(n+\tau)-\mu_z\right)^H\right]$$

Edit Note There is a correction to this definition as applied to signal-processing, see Matt's Answer below.

The covariance does not involve a concept of time, it assumes each element of the random vector is a different realization of some random generator. The autocorrelation assumes a random vector is the time evolution of some initial random generator. Yet in the end, they are both the same mathematical entity, a sequence of numbers. If you let $X=Y=Z$, then it appears $$C_{XY}=R_{ZZ}$$ Is there something more subtle that I am missing?

$\endgroup$
  • $\begingroup$ The definition of AutoCorrelation $R_{ZZ}(\tau)$ is incorrectly stated in the question as pointed out by Matt $\endgroup$ – ijuneja Oct 20 '18 at 6:07
11
$\begingroup$

According to your definition of autocorrelation, the autocorrelation is simply the covariance of the two random variables $Z(n)$ and $Z(n+\tau)$. This function is also called autocovariance.

As an aside, in signal processing, the autocorrelation is usually defined as

$$R_{XX}(t_1,t_2)=E\{X(t_1)X^*(t_2)\}$$

i.e., without subtracting the mean. The autocovariance is given by

$$C_{XX}(t_1,t_2)=E\{[X(t_1)-\mu_X(t_1)][X^*(t_2)-\mu^*_X(t_2)]\}$$

These two functions are related by

$$C_{XX}(t_1,t_2)=R_{XX}(t_1,t_2)-\mu_X(t_1)\mu^*_X(t_2)$$

$\endgroup$
  • $\begingroup$ If you look at $\tau$ as a variable, then the autocorrelation becomes a function of that "time gap" which can yield very interesting information about the data set. Look at the relation between autocorrelation, discrete Fourier transforms and the Wiener–Khinchin theorem. $\endgroup$ – PhilMacKay Feb 28 '17 at 16:36
  • $\begingroup$ @PhilMacKay: Sure, but that only works for WSS processes. I gave the definitions for the general case, where processes are not necessarily stationary. $\endgroup$ – Matt L. Feb 28 '17 at 16:43
  • $\begingroup$ Yes indeed non-stationary processes can be annoying for data analysis, which is why I always try to de-trend a data before using my beloved statistical tools! It's not always possible, though... $\endgroup$ – PhilMacKay Feb 28 '17 at 20:05
0
$\begingroup$

Notice how your definition of Autocorrelation includes an additional term $\tau$, which specifies an offset from the two sequences of number $Z(n)$ and $Z(n+\tau)$. In fact, the notation suggest that $R_{ZZ}(\tau)$ is a continuous function defined for any $\tau \in \mathbb{R}^+$, while $C_{XY}$ is a scalar.

As you mentioned, if you let $X=Y=Z$, then you are implying that $\tau = 0$, which is one special case of $R_{ZZ}(\tau)$.

In my personal experience (astrophysics, various sensor processing), the covariance was used as a coefficient to check the similarity of two datasets, while the autocorrelation was used to characterize the correlation distance, that is, how quickly a data evolves to become another data entirely.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.