# Covariance matrix of an adaptive filter input

I run many times in equations containing the trace of covariance matrix of an adaptive filter input. But it is not really clear what it is.

For example in this paper the input covariance matrix is

$$\textbf{R} = E\left(\textbf{u}_i^*\textbf{u}_i \right)$$

where:

• $\textbf{u}_i$ is row vector of input (seems to be regression vector in given time)

• $^*$ denotes Hermitian conjugation (complex conjugation for scalars)

• $E$ is not explained, but I would expect it is the expected value

Questions:

1. What does it mean?
2. How does the matrix look like, what is the shape of it?
3. How do I estimate it from input data $\textbf{U}$ - matrix where one row represents one input vector?
• wikipedia has an article on covariance matrices. I really don't understand the question – from the formula alone, it's clear that its shape is quadratic, and honestly: I think you can come up with a way of estimating an expectation value from multiple observations. – Marcus Müller Mar 6 '17 at 13:34
• @MarcusMüller Thanks for reply. If I understand you correctly, then in case of fllter size 10 and data of size 1000 input vectors, I should create 1000 matrices (of size 10x10) and estimate mean value of them (10x10 result)? – matousc Mar 6 '17 at 17:06
• sounds like a good estimator :) – Marcus Müller Mar 6 '17 at 17:07
• @MarcusMüller If this is correct answer, then please make an proper answer so I can accept it. I do not think that I am the only one who do not find this solution obvious. – matousc Mar 6 '17 at 17:09

The Covariance Matrix is commonly defined as

$$\mathbf Q = E\left[ (\mathbf x -\mathbf\mu_{x})(\mathbf x -\mathbf\mu_{x})^*\right]$$ with $\mu$ denoting the mean value, i.e. $\mu_{x}=E\left[\mathbf x\right]$, and $\mathbf x$ being column vectors. The fact that you define the covariance matrix as

$$\mathbf{R}_i = E\left[\textbf{u}_i^*\textbf{u}_i \right]$$

indicates that your $\mathbf u_i$ have zero mean and are row vectors; that might be a result of the signal model you use, but it's pretty non-standard.

Furthermore, the product of a column vector with its hermitian is inherently hermitian (symmetric if real).

Now, one way to estimate an expectation value from multiple observations. Wikipedia has an article on estimating covariance matrices, so here's just the gist: the most intuitive estimator is the sample covariance matrix; adapted to your formula, that'd be

$$\hat{\mathbf R} = \frac1{N-1}\sum\limits_{i=1}^N \mathbf u_i^*\mathbf u_i$$

• I think if you know that the mean value is zero, then the proper formula is $\hat{\mathbf{R}} = \frac{1}{N}\sum_{i=1}^N \mathbf{u}_i^* \mathbf{u}_i$. The division by $N-1$ comes about when you need to estimate the mean value from the data as well. – hops Mar 6 '17 at 22:09