Covariance vs Autocorrelation

Question

I'm trying to figure out if there is a direct relationship between these concepts. Strictly from the definitions, they appear to be different concepts in general. The more I think about it, however, the more I think they are very similar.

Let $X,Y$ be WSS random vectors. The covariance, $C_{XY}$, is given by $$C_{XY}=E\left[(X-\mu_x)(Y-\mu_y)^H\right]$$ where $H$ stands for the Hermitian of the vector.

Let $Z$ be a WSS random vector. The autocorrelation function, $R_{XX}$, is given by $$R_{ZZ}(\tau)=E\left[\left(Z(n)-\mu_z\right)\left(Z(n+\tau)-\mu_z\right)^H\right]$$

Edit Note There is a correction to this definition as applied to signal-processing, see Matt's Answer below.

The covariance does not involve a concept of time, it assumes each element of the random vector is a different realization of some random generator. The autocorrelation assumes a random vector is the time evolution of some initial random generator. Yet in the end, they are both the same mathematical entity, a sequence of numbers. If you let $X=Y=Z$, then it appears $$C_{XY}=R_{ZZ}$$ Is there something more subtle that I am missing?

Answer 1

According to your definition of autocorrelation, the autocorrelation is simply the covariance of the two random variables $Z(n)$ and $Z(n+\tau)$. This function is also called autocovariance.

As an aside, in signal processing, the autocorrelation is usually defined as

$$R_{XX}(t_1,t_2)=E\{X(t_1)X^*(t_2)\}$$

i.e., without subtracting the mean. The autocovariance is given by

$$C_{XX}(t_1,t_2)=E\{[X(t_1)-\mu_X(t_1)][X^*(t_2)-\mu^*_X(t_2)]\}$$

These two functions are related by

$$C_{XX}(t_1,t_2)=R_{XX}(t_1,t_2)-\mu_X(t_1)\mu^*_X(t_2)$$

Answer 2

Notice how your definition of Autocorrelation includes an additional term $\tau$, which specifies an offset from the two sequences of number $Z(n)$ and $Z(n+\tau)$. In fact, the notation suggest that $R_{ZZ}(\tau)$ is a continuous function defined for any $\tau \in \mathbb{R}^+$, while $C_{XY}$ is a scalar.

As you mentioned, if you let $X=Y=Z$, then you are implying that $\tau = 0$, which is one special case of $R_{ZZ}(\tau)$.

In my personal experience (astrophysics, various sensor processing), the covariance was used as a coefficient to check the similarity of two datasets, while the autocorrelation was used to characterize the correlation distance, that is, how quickly a data evolves to become another data entirely.