# Difference between $\mathbb{E}[\mathbf{x} \mathbf{x}^{\rm{H}}]$ and $\mathbb{E}[(\mathbf{x}-\boldsymbol{\mu}) (\mathbf{x}-\boldsymbol{\mu})^{\rm{H}}]$

Let us have a random vector $\mathbf{x} \sim \mathcal{CN} (\boldsymbol{\mu}, \boldsymbol{\Sigma})$ with $\boldsymbol{\mu} \neq \mathbf{0}$. What can we say about the relationship between the elements of $\mathbf{x}$ in the following two separate cases?

1) If $\boldsymbol{\Sigma} \triangleq \mathbb{E}[(\mathbf{x}-\boldsymbol{\mu}) (\mathbf{x}-\boldsymbol{\mu})^{\rm{H}}]$ is a diagonal matrix;

OR

2) If $\mathbf{R} \triangleq \mathbb{E}[\mathbf{x} \mathbf{x}^{\rm{H}}] = \boldsymbol{\mu} \boldsymbol{\mu}^{\rm{H}} + \boldsymbol{\Sigma}$ is a diagonal matrix.



In 1), the elements of $\mathbf{x}$ are all independent. But what is the interpretation of 2)?

• So, what have you considered so far? What does it mean that a matrix is diagonal for the non-diagonal entries, and what does the value of these entries mean for the vectors? Exactly the same as for the $-\mathbf \mu$ case, right. You just might want to write $\mathbf x$ as sum as a zero-mean RV and a constant! – Marcus Müller Nov 24 '17 at 20:31
• If $\mu \neq \boldsymbol{0}$ then $\boldsymbol{R} = \mathbb{E} \left[ \boldsymbol{x} \boldsymbol{x}^{H} \right]$ can not be diagonal. – Royi Nov 24 '17 at 22:39
• @MarcusMüller If the $(i,j)$th entry of $\boldsymbol{\Sigma}$ is zero, it means that the $i$th element and the $j$th element of $\mathbf{x}$ are independent (i.e., they vary independently around the corresponding elements of $\boldsymbol{\mu}$). On the other hand, if the $(i,j)$th entry of $\mathbf{R} = \boldsymbol{\Sigma} + \boldsymbol{\mu} \boldsymbol{\mu}^{\rm{H}}$ is zero, does it mean that the $i$th element and the $j$th element of $\mathbf{x} + \boldsymbol{\mu}$ are independent? If yes, does this imply that the elements of $\mathbf{x}$ alone are independent? – TheDon Nov 24 '17 at 23:07
• @Royi Yes, it can if the off-diagonal elements of $\boldsymbol{\mu} \boldsymbol{\mu}^{\rm{H}}$ are $-$ the off-diagonal elements of $\boldsymbol{\Sigma}$. – TheDon Nov 24 '17 at 23:11
• Let me be more accurate. If there are $i \neq i$ such that ${\mu}_{i} \neq 0$ and ${\mu}_{j} \neq 0$ then $\boldsymbol{R}$ can't be diagonal. – Royi Nov 24 '17 at 23:44

1. For the first case, as you wrote, it means the elements are not correlated. Since this is a Gaussian Random Vector it means the elements are independent.
2. It means that at most only one element of $\boldsymbol{\mu}$ is not zero.
Since if there were more than 1, the matrix $\boldsymbol{R}$ wasn't diagonal.

### Update

Let's define $\hat{\boldsymbol{x}} = \boldsymbol{x} - \boldsymbol{\mu}$, namely its centered version.

Looking ta the correlation matrix elements:

\begin{align*} \boldsymbol{R}_{i, j} & = \mathbb{E} \left[ \boldsymbol{x}_{i} \boldsymbol{x}_{j} \right] = \mathbb{E} \left[ \left( \hat{\boldsymbol{x}}_{i} + \boldsymbol{\mu}_{i} \right) \left( \hat{\boldsymbol{x}}_{j} + \boldsymbol{\mu}_{j} \right) \right] \\ & = \mathbb{E} \left[ \hat{\boldsymbol{x}}_{i} \hat{\boldsymbol{x}}_{j} \right] + \boldsymbol{\mu}_{j} \mathbb{E} \left[ \hat{\boldsymbol{x}}_{i} \right] + \boldsymbol{\mu}_{i} \mathbb{E} \left[ \hat{\boldsymbol{x}}_{j} \right] + \boldsymbol{\mu}_{i} \boldsymbol{\mu}_{j} \\ & = \Sigma_{i, j} + \boldsymbol{\mu}_{i} \boldsymbol{\mu}_{j} \end{align*}

Now it is really easy to see what is needed for this element to vanish.

• Assume we have $\mathbf{y} \sim \mathcal{CN} (\bar{\boldsymbol{\mu}}, \bar{\boldsymbol{\Sigma}})$: then, $\mathbb{E}[\mathbf{y} \mathbf{y}^{\rm{H}}] = \bar{\boldsymbol{\mu}} \bar{\boldsymbol{\mu}}^{\rm{H}} + \bar{\boldsymbol{\Sigma}} = \bar{\mathbf{U}} \bar{\mathbf{D}} \bar{\mathbf{U}}^{\rm{H}}$ (after applying the eigenvalue decomposition). Hence, if we set $\mathbf{x} = \bar{\mathbf{U}}^{\rm{H}} \mathbf{y}$, its mean is $\boldsymbol{\mu} = \bar{\mathbf{U}}^{\rm{H}} \bar{\boldsymbol{\mu}}$ (where all entries can be $\neq 0$) and $\mathbb{E}[\mathbf{x} \mathbf{x}^{\rm{H}}] = \bar{\mathbf{D}}$. – TheDon Nov 25 '17 at 8:27
• The above, is a case that shows that you can have a diagonal $\mathbf{R}$ for a certain $\mathbf{x}$. Regardless of how such $\mathbf{x}$ is obtained, I would like to understand the "physical meaning" of this case (e.g., to what extent the elements of $\mathbf{x}$ are uncorrelated/independent). – TheDon Nov 25 '17 at 8:38
• I might miss something here but it doesn't sounds right. How could 2 random variable which are not centered be un correlated? If they have DC part they will be correlated. Here correlation is not the centered correlation. Again, if the vector has 2 variable which has different than zero mean they must be correlated hence the corresponding elelment in $\boldsymbol{R}$ will be different than zero. – Royi Nov 25 '17 at 9:40
• I understand your doubts, and that is why I decided to open this thread. But if you try to implement what I wrote in the above comment on Matlab, you will find this to be true (I can give you a small code in Matlab by private message if you want). The problem here is that I cannot give a "physical meaning" to this. – TheDon Nov 25 '17 at 10:22
• What do you mean exactly by "Here correlation is not the centered correlation"? – TheDon Nov 25 '17 at 11:40

One must be careful when asking questions about the relationships between the elements of a complex random vector.

The short answer to your question is that you cannot say much for either cases simply by considering the covariance (or correlation) matrix.

Actually, the covariance (correlation) matrix is not enough to capture all the relationships that exist between the elements of a complex random vector. For that we need to also consider the so-called pseudo-covariance (pseudo-correlation) matrix.

The idea is to consider real and imaginary parts of the complex random vector elements and all possible relationships between them.

## Explanation

Let us consider, for simplicity, a zero-mean complex random vector:

\begin{align} \mathbf{z}=\mathbf{x}+j\mathbf{y} &= \begin{bmatrix} z_1 \\ \vdots \\ z_n \end{bmatrix}= \begin{bmatrix} x_1+jy_1 \\ \vdots \\ x_n+jy_n \end{bmatrix}. \end{align}

Let us consider two different elements $z_k$ and $z_l$ of $\mathbf{z}$: \begin{matrix} z_k = x_k+jy_k \\ z_l = x_l+jy_l, \end{matrix} then, we have the following relationships we need to consider:

• Auto (A) relationships: between $x_k, y_k, x_l, y_l$ and themselves, respectively.
• Horizontal (H) relationships: between $x_k$ and $y_k$, and between $x_l$ and $y_l$.
• Vertical (V) relationships: between $x_k$ and $x_l$, and between $y_k$ and $y_l$.
• Diagonal (D) relationships: between $x_k$ and $y_l$, and between $x_l$ and $y_k$.

Note that we want all existing relationships between all real and imaginary parts.

The covariance matrix of $\mathbf{z}$ is defined as $\mathbf{C}_{zz}=\mathbb{E}[(\mathbf{z}-\boldsymbol{\mu})(\mathbf{z}-\boldsymbol{\mu})^H]$, which is equal to the correlation matrix $\mathbf{R}_{zz}=\mathbb{E}[\mathbf{z}\mathbf{z}^H]$ since we supposed $\boldsymbol{\mu}=\boldsymbol{0}$.

If we develop we get:

$$\mathbf{R}_{zz}=\mathbb{E}[\mathbf{z}\mathbf{z}^H]=\mathbb{E}[(\mathbf{x}+j\mathbf{y})(\mathbf{x}+j\mathbf{y})^H] = \mathbb{E}[(\mathbf{x}+j\mathbf{y})(\mathbf{x}-j\mathbf{y})^T] \\ = \mathbb{E}[\mathbf{x}\mathbf{x}^T]+ \mathbb{E}[\mathbf{y}\mathbf{y}^T] + j(\mathbb{E}[\mathbf{y}\mathbf{x}^T] - \mathbb{E}[\mathbf{x}\mathbf{y}^T])$$ $$= \mathbf{R}_{xx}+\mathbf{R}_{yy} + j(\mathbf{R}_{xy}^T-\mathbf{R}_{xy}) \tag{I}$$ $$=\mathbb{E}\begin{bmatrix} x_1^2 & \cdots & x_1x_n\\ \vdots & \ddots & \vdots\\ x_nx_1 & \cdots & x_n^2 \end{bmatrix} + \mathbb{E}\begin{bmatrix} y_1^2 & \cdots & y_1y_n\\ \vdots & \ddots & \vdots\\ y_ny_1 & \cdots & y_n^2 \end{bmatrix} +j\left( \mathbb{E}\begin{bmatrix} x_1y_1 & \cdots & x_ny_1\\ \vdots & \ddots & \vdots\\ x_1y_n & \cdots & x_ny_n \end{bmatrix} - \mathbb{E}\begin{bmatrix} x_1y_1 & \cdots & x_1y_n\\ \vdots & \ddots & \vdots\\ x_ny_1 & \cdots & x_ny_n \end{bmatrix} \right ) \\ =\mathbb{E}\begin{bmatrix} x_1^2 & \cdots & x_1x_n\\ \vdots & \ddots & \vdots\\ x_nx_1 & \cdots & x_n^2 \end{bmatrix} + \mathbb{E}\begin{bmatrix} y_1^2 & \cdots & y_1y_n\\ \vdots & \ddots & \vdots\\ y_ny_1 & \cdots & y_n^2 \end{bmatrix} +j \mathbb{E}\begin{bmatrix} 0 & \cdots & (x_ny_1-x_1y_n)\\ \vdots & \ddots & \vdots\\ (x_1y_n-x_ny_1) & \cdots & 0 \end{bmatrix}.$$

What we end up having is: (A) and (V) relationships from $\mathbf{R}_{xx}$ and $\mathbf{R}_{yy}$, and (D) relationships from $\mathbf{R}_{xy}$, BUT we loose the (H) relationships.

To compensate for that, and for the fact that both $\mathbf{R}_{xx}$ and $\mathbf{R}_{yy}$ define the real-part of $\mathbf{R}_{zz}$, we define the pseudo-covariance matrix of $\mathbf{z}$ as $\mathbf{\bar{C}}_{zz}=\mathbb{E}[(\mathbf{z}-\boldsymbol{\mu})(\mathbf{z}-\boldsymbol{\mu})^T]$, and similarly the pseudo-correlation matrix $\mathbf{\bar{R}}_{zz}=\mathbb{E}[\mathbf{z}\mathbf{z}^T]$, by simply replacing the Hermitian transpose with a transpose.

With a similar development, to the above one, we end up having: $$\mathbf{\bar{R}}_{zz} = \mathbf{R}_{xx}-\mathbf{R}_{yy} + j(\mathbf{R}_{xy}^T+\mathbf{R}_{xy}) \tag{II}\\ =\mathbb{E}\begin{bmatrix} x_1^2 & \cdots & x_1x_n\\ \vdots & \ddots & \vdots\\ x_nx_1 & \cdots & x_n^2 \end{bmatrix} - \mathbb{E}\begin{bmatrix} y_1^2 & \cdots & y_1y_n\\ \vdots & \ddots & \vdots\\ y_ny_1 & \cdots & y_n^2 \end{bmatrix} +j \mathbb{E}\begin{bmatrix} 2x_1y_1 & \cdots & (x_ny_1+x_1y_n)\\ \vdots & \ddots & \vdots\\ (x_1y_n+x_ny_1) & \cdots & 2x_ny_n \end{bmatrix}.$$ So, we get here all relationships (A), (H), (V), and (D). Nonetheless, we cannot extract them unless we use jointly the covariance and pseudo-covariance matrices. For example, if we want (A) and (V) relationships of $\mathbf{x}$ we add $(I)$ and $(II)$ and consider the real part of the result.

## The special, simple, case of circular complex random vectors

A simple case to deal with is the case of a circular complex random vector $\mathbf{z}$ for which we have $\mathbf{\bar{C}_{zz}}=\boldsymbol{0}$, otherwise we say that the complex random vector is noncircular.

From $(II)$ (replacing $\mathbf{R}$ with $\mathbf{C}$) we can see that $\mathbf{\bar{C}_{zz}}=\boldsymbol{0}$ implies: $$\mathbf{C}_{xx}=\mathbf{C}_{yy} \tag{*}$$ $$\mathbf{C}_{xy}=-\mathbf{C}_{xy}^T \tag{**}$$

Equation $(*)$ indicates (considering diagonal elements) that for each element of $\mathbf{z}$ the variance of the real part $x_i$ is equal to the variance of the imaginary part $y_i$, $i=1,\dots, n$, i.e. $\mathbb{E}[(x_i-\mu_{x_i})^2]=\mathbb{E}[(x_i-\mu_{x_i})^2]$ (it helps to relate the name circular to equal variance on the real and imaginary axes, which gives points distributed inside a circle!). Moreover covariances are equal, i.e. $\mathbb{E}[(x_i-\mu_{x_i})(x_j-\mu_{x_j})]=\mathbb{E}[(y_i-\mu_{y_i})(y_j-\mu_{y_j})]$, $i\ne j$.

Equation $(**)$ shows that $\mathbf{C}_{xy}$ is skew-symmetric, which implies null diagonal elements, i.e. $\mathbb{E}[(x_i-\mu_{x_i})(y_i-\mu_{y_i})]=0$, indicating that $x_i$ and $y_i$ are uncorrelated.

## Back to the OP questions

### What relationships exist between elements of $\mathbf{z}$ if $\mathbf{C}_{zz}$ is (complex) diagonal?

We know now that we have to know something about $\mathbf{\bar{C}}_{zz}$ too.

If we suppose circular complex random variables then we have from $(I), (*)$, and $(**)$:

$$\mathbf{C}_{zz}=2\mathbf{C}_{xx}-2j\mathbf{C}_{xy}=2\mathbf{C}_{xx}=\text{diag}(\alpha_1, \dots, \alpha_n),$$ with skew-symmetric $\mathbf{C}_{xy}=\boldsymbol{0}$, due to the diagonal structure since the only non-zero elements (off diagonal) are now zero.

Finally, we have:

• (H) relationships of $\mathbf{x}$ and $\mathbf{y}$ are uncorrelatedness (circular case).
• (D) relationships of $\mathbf{x}$ and $\mathbf{y}$ are uncorrelatedness ($\mathbf{C}_{xy}=\boldsymbol{0}$).
• (A) relationships of $\mathbf{x}$ are the same as (A) relationships of $\mathbf{y}$ (circular case) and are such that $\mathbb{E}[(x_i-\mu_{x_i})^2]=\alpha/2$ (diagonal matrix).
• (V) relationships of $\mathbf{x}$ are the same as (V) relationships of $\mathbf{y}$ (circular case) and are uncorrelatedness (diagonal matrix).

If we suppose noncircular complex random variables then all we know is:

• (D) relationships of $\mathbf{x}$ and $\mathbf{y}$ are uncorrelatedness.
• (V) relationships of $\mathbf{x}$ and $\mathbf{y}$ are uncorrelatedness.
• $\mathbf{C}_{xx}+\mathbf{C}_{yy} = \text{diag}(\alpha_1, \dots, \alpha_n)$ and $\mathbf{C}_{xy}^T-\mathbf{C}_{xy} = \text{diag}(\beta_1, \dots, \beta_n)$ (with $\mathbf{C}_{zz}=\text{diag}(\alpha_1, \dots, \alpha_n)+j\text{diag}(\beta_1, \dots, \beta_n)$).

Hence, to know more about the relationships we need the computation of the pseudo-covariance matrix.

### What relationships exist between elements of $\mathbf{z}$ if $\mathbf{R}_{zz}$ is (complex) diagonal?

If $\boldsymbol{\mu}\ne \boldsymbol{0}$, we have: $$\mathbf{C}_{zz}=\mathbf{R}_{zz}-\boldsymbol{\mu}\boldsymbol{\mu}^H.$$

If at least two elements of $\boldsymbol{\mu}$ are non-zero then $\boldsymbol{\mu}\boldsymbol{\mu}^H$ is non-diagonal (since the off-diagonal elements are products of complex numbers which cannot be zero if the elements are non-zero), and so will be $\mathbf{C}_{zz}$. Then, considering this non-diagonal structure and, again, using $\mathbf{\bar{C}}_{zz}$, we can conduct a similar analysis as the one conducted above.

• Wow... You really put into this a lot of effort. Great! +1. – Royi Apr 7 '18 at 16:41
• Thanks @Royi Hope it's helpful and that it correctly answers the question. – Learn_and_Share Apr 8 '18 at 8:03