Every time I think I have understood the covariance matrix, someone else comes up wih a different formulation.

I am currently reading this paper:

J. Benesty, "Adaptive eigenvalue decomposition algorithm for passive acoustic source localization", J. Acoust. Soc. Am. Volume 107, Issue 1, pp. 384-391 (2000)

and I have come across a formulation I do not quite understand. Here, the author is constructing the covariance matrix between two signals, $x_1$, and $x_2$. Those two signals are from different sensors.

For the covariance matrix of one signal, I know that we can get it by calculating the regression matrix, and then multiply it by the Hermitian of that same matrix, and dividing by $N$, the length of the original vector. The size of the covariance matrix here can be arbitrary, with maximum size being $N\times N$.

For the covariance matrix of two spatial signals, if we place the first signal in the first row, and the second signal in the second row of a matrix, then multiply by its Hermitian, and also divide by $N$, then we get a $2\times 2$ covariance matrix of both spatial signals.

However, in this paper, the author computes what looks like four matricies, $R_1{}_1, R_1{}_2, R_2{}_1$, and $R_2{}_2$, and then puts them into a super matrix and calls that the covariance matrix.

Why is this so? Here is an image of the text:

enter image description here


1 Answer 1


If you have two signal vectors $x_1[n]$ and $x_2[n]$ each of $N$ elements, then there are two different things we can consider.

  1. How do the quantities $\displaystyle\sum_{n=1}^N x_i[n]x_j[n],~ i, j \in \{1,2\}$ compare? In particular, when the signals are noisy and the noises can be considered to be jointly stationary (or jointly wide-sense stationary), these quantities can be used to estimate the noise variances in the two signals as well as the covariance of the noises at any fixed sampling time. This is what you get from the $2\times 2$ covariance matrix $$R_{2\times 2} = \left[\begin{matrix} \sigma_1^2 & C\\ C & \sigma_2^2\end{matrix}\right].$$ The noise in $x_1[n]$ has variance $\sigma_1^2 = R_{1,1}$ which might be different from $R_{2,2} = \sigma_2^2$, the variance of the noise in $x_2[n]$. But the noises are correlated with covariance $R_{1.2}=R_{2,1} = C$. Now, if we plan on doing things with just what happens at $n$, ignoring whatever might be happening at $n-1$ or $n+1$ etc., then this is all the information we need.

  2. Unless the noise is known to be (or assumed to be) white noise so that noise samples from different sampling instants are independent (and therefore uncorrelated) or we simply assume uncorrelated noise samples, there is information that we are ignoring by not considering the correlation between $x_1[n]$ and $x_1[m]$, samples from the same process at different times or locations, and the correlation between $x_1[n]$ and $x_2[m]$, samples from the two processes at different times or locations. This additional information might lead to a better estimate/solution. Now we have a total of $2N$ noise samples, and therefore a $2N \times 2N$ covariance matrix to consider. If we arrange matters the way the authors did, we have $R_{\text{full}} = E[XX^T]$ where $$X = (x_1[1],x_1[2],\ldots,x_1[N],x_2[1],x_2[2],\ldots,x_2[N])^T = (\mathbf x_1,\mathbf x_2)^T$$ and so $$R_{\text{full}} = \left[ \begin{matrix}R_{\mathbf x_1,\mathbf x_1} & R_{\mathbf x_1,\mathbf x_2}\\ R_{\mathbf x_2,\mathbf x_1} & R_{\mathbf x_2,\mathbf x_2}\end{matrix} \right]$$ where $R_{\mathbf x_i,\mathbf x_j} = E[\mathbf x_i\mathbf x_j^T]$. Note that $R_{\mathbf x_i,\mathbf x_j}$ is, in essence, the cross-correlation function of $(x_i[1],x_i[2],\ldots,x_i[N])$ and $(x_j[1],x_j[2],\ldots,x_j[N])$ if $i \neq j$ and the autocorrelation function if $i = j$. If the noise processes are white and uncorrelated except when $n = m$, then $$R_{\text{full}} \rightarrow R_{\text{simple}} = \left[ \begin{matrix}\sigma_1^2I & CI\\ CI & \sigma_2^2I \end{matrix}\right]$$ where $I$ is the $N\times N$ identity matrix, and $\sigma_1^2, \sigma_2^2$ and $C$ are as defined in Item 1 above. How realistic this noise model might be is something for the end user to determine. If the model is realistic, then nothing is gained by looking at the $2N\times 2N$ matrix $R_{\text{full}}$ since all the information is there in the $2\times 2$ matrix $R_{2\times 2}$ of Item 1 above. Ditto if the model is unrealistic but we don't intend to (or are unable to) use all the information in the full $2N\times 2N$ matrix $R_{\text{full}}$; we will make do with just $\sigma_1^2, \sigma_2^2$ and $C$ of Part 1 for which we don't need $R_{\text{full}}$ or $R_{\text{simple}}$, just $R_{2\times 2}$.

  • $\begingroup$ Thanks. First, shouldn't the sigma in (1) say from n = 0 to N-1? (Not from i = 1 to n). $\endgroup$
    – Spacey
    Commented Jan 2, 2012 at 22:32
  • $\begingroup$ I am not sure I still understand what/why we are doing it this way. Are you saying that for (1), since the noises in both vectors are completely independent of each other, we have to use that method, and thus get a 2x2 co-variance matrix, but that in the second case (2), since the noises in the vectors are not independent, we have to concatenate both vectors and then compute their co-variance matrix? Why though? Im afraid I still do not understand the motivation here... $\endgroup$
    – Spacey
    Commented Jan 2, 2012 at 22:35
  • $\begingroup$ Thanks I will read it again. Also, the subscript for sigma must be 'n', not 'i'. $\endgroup$
    – Spacey
    Commented Jan 3, 2012 at 1:22
  • $\begingroup$ I will jot down some more questions/comments tomorrow, but for now, what are the 'official' names of $R_2{}_x{}_2, R_{\text{full}}$, and $R_{\text{simple}}$? I cant imagine they are all called 'co-variance matrices', since this leads to confusion, (as was the main motivation for this question). What are they normally referred to? $\endgroup$
    – Spacey
    Commented Jan 3, 2012 at 7:37
  • $\begingroup$ 1) "can be considered to be jointly stationary (or jointly wide-sense stationary)" Here, do you mean if $x_1$ and $x_2$ are both independent? 2) "estimate the noise variances in the two signals as well as the covariance of the noises at any fixed sampling time. " What do you mean 'at any fixed sampling time' here? In order to compute the 2x2, we are using all time samples of both signals... 3) "Now we have a total of 2N noise samples," I guess I do not understand by what 'right' we can simply concatenate two spatial signals like that. Why are we 'allowed' to do that? $\endgroup$
    – Spacey
    Commented Jan 4, 2012 at 2:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.