# What is a covariance matrix?

Suppose you have k samples from each of the N elements of a uniform linear array (ULA) of sensors:

1. What is the physical meaning of a covariance matrix?
2. How do you form a covariance matrix with the samples?
3. How do you decide how many samples you need to use to form the covariance matrix?

It's the key point of array signal processing, I suppose. Say $x$ is the input vector of $[N,1]$ dimension collected from $N$ array sensors. $x(k)$ is its realization at the $k$ moment of time. By its definition covariance matrix (sometimes it's called autocorrelation matrix):

• $R = E[x\cdot x^H]$ ,

where $E[]$ is expectation operator and $x^H$ is Hermitian conjugate. For the ergodic process

• $R = \lim_{M\to\infty} 1/M \cdot \displaystyle\sum_{k=0}^M x(k) \cdot [x(k)]^H$ .

But in practice we can estimate $R$ with necessary precision with the snapshot of finite length. In the array processing theory snapshot is a group of vectors $x(k)$. It's the basic data block for array processing algorithms.

• $R = 1/K \cdot \displaystyle\sum_{k=0}^K x(k) \cdot [x(k)]^H$ ,

where $K$ is the number of spatial vectors or snapshot size. The interesting question is what is optimal value for $K$. In models I've done, $K$ varies from 64 to 256. The estimation precision is enough for MVDR beamformer or Capon spectral estimation. There is interesting trick for adaptive beamformer design (if you have further interest). You can reduce $K$ (and accelerate adaptation process) if you use so-called diagonal loading, look:

• $R = R + \sigma^2 \cdot I$,

where $\sigma$ is some variable depending on SNR (maybe someone define it more precisely?) and $I$ is identity matrix of the size $[N, N]$, same size as $R$ actually. But if you want to do spectral estimation procedure by your array (DoA estimation), you shouldn't perform diagonal loading because it will rise noise floor of the estimation and some weak signals of interest will be lost.

Covariance matrix is the second order statistic of the random process which is measured at the array sensors. It contains information about the sources in space (number, strength, direction) and can be used for sources detection and separation. Actually the number of independent spatial signals at the input of array and the rank of $R$ is the same. Singular values decomposition (SVD) of $R$ gives the information about signal subspace which is necessary for subspace-based DoA estimation techniques like MUSIC or ESPRIT. It is worth to say algorithms based upon covariance matrix inversion/decomposition suffers from source correlation. If the sources is highly correlated (have the same waveforms or too close direction) they can't be separated. And also the system performance degrades in highly correlated scenario.

Very good reference about the topic will be

Harry L. Van Trees - Detection, Estimation, and Modulation Theory, Optimum Array Processing

In the book more general form of covariance matrix is discussed. It's shown that it can be defined either in time domain or in frequency domain. And furthermore frequency domain interpretation is more common since it allows us to solve the problem of wideband beamformer (or DoA estimation) very easy.

Hope this helps.

• Do the rows or columns of the covariance matrix have any direct physical meaning relating to radiating the source(s) or do you have do the eigenvector decomposition to gain insight in the radiating source(s)? – random_dsp_guy Aug 15 '14 at 4:23
• The rows and columns of the matrix are part of autocorrelation function of the spatial signal mixture. That's a pity but all "tasty" information is contained in its inverse or eigen values. So this is of practical challenge. Furthermore since some signals can be very strong and some - very weak, the problem of singularity arises. So for practical computations double precision or long double is the case. – Serj Aug 15 '14 at 4:29
• Can you offer more details as to why "algorithms based upon covariance matrix inversion/decomposition suffers from source correlation"? – random_dsp_guy Aug 20 '14 at 2:42
• Suppose 2 signals of the same waveform (e.g. LFM) that are coming from different angle to your array. First set that there is no time delay between these signals. Construct plane waves and estimate covariance matrix. Then compute eigen values via $svd(R)$. How much non-noise eigen values will you see? Then start to increase time delay between your signal sources and also look at eigen values of their spatial covariance matrix. - What will happen with them? Now suppose 2 different waveforms and do the same with the angle of arrival. Try this one time in your model and it will be clear. – Serj Aug 20 '14 at 2:50
• is the covariance matrix always the autocorrelation matrix? – BigBrownBear00 Dec 13 '18 at 21:12