# MLE parameter estimation -- confusion regarding some terms in the pdf of complex normal r.v (Part 2)

This question is based on the application of the pdf which was an earlier question of mine asked here Confusion regarding pdf of circularly symmetric complex gaussian rv

If $v \sim CN(0,2\sigma^2_v)$ is a circularly complex Gaussian random variable which acts as the measurement noise in this model $$y_n = A + v_n \tag{1}$$ where $y$ is the observation and $A$ is a scalar unknown value which needs to be estimated. I am having a slight confusion whether there will be a 2 in the denominator of Eq(3) and Eq(4) with the $\exp(.)$ term. Based on the answer in the link, there should be no sqrt term with $\pi$ in the denominator, if $v \sim CN(0,2\sigma^2_v)$. If $v \sim N(0,\sigma^2_v)$ then there is a sqrt term.

Can somebody please check if I have correctly written out the log-likelihood? I think I am missing a 2 in the denominator of $\exp[.]$ term in Eq(3) but I am not quite sure.

Thank you for your time and help.

$$P_y(y_1,y_2,...,y_N) = \prod_{n=1}^N\frac{1}{2\pi \sigma^2_v} \exp \bigg(\frac{-{({y_n-A})}^H ({y_n-A})}{2\sigma^2_v} \bigg) \tag{2}$$

taking log $$\ell = -N\ln(2\pi\sigma^2_v) - \frac{1}{\sigma^2_v} {\bigg[{[\sum_{n=1}^{N} {(y_n - A)}{(y_n - A)}^{\mathsf{H}} ]}\bigg]}. \tag{3}$$ $$= -N\ln(2\pi\sigma^2_v)- \frac{1}{2 \sigma^2_v}{\bigg[ \sum_{n=1}^{N}y_n y_n^\mathsf{H} - 2 \sum_{n=1}^N y_n A\bigg]} - \frac{1}{2 \sigma^2_v}{\bigg[ \sum_{n=1}^N {AA}^\mathsf{H} ] \bigg]} \tag{4}$$

• Going from (2) to (3), why did $(y-A)^H(y-A)$ turn into $(y-A)(y-A)^H$? Mar 19, 2018 at 1:54
• @AtulIngle: I used the commutative law in multiplication: A.B = B.A Mar 19, 2018 at 2:04
• Ah, so $(\cdot)^H$ really means complex conjugate here because these are complex scalars. In general, this is not true for vectors/matrices. Mar 19, 2018 at 2:05

I looked this up on Wikipedia: A complex Gaussian random variable $V = \mathfrak{Re}(V)+j\mathfrak{Im}(V)$ is said to be zero mean circularly symmetric $\mathcal{CN}(0,\Gamma)$ if the random vector $[\mathfrak{Re}(V),\mathfrak{Im}(V)]$ is a Gaussian random vector with mean $[0,0]$ and covariance matrix $\frac{1}{2}\begin{bmatrix} \mathfrak{Re}(\Gamma) & -\mathfrak{Im}(\Gamma) \\ \mathfrak{Im}(\Gamma) & \mathfrak{Re}(\Gamma) \end{bmatrix} = \frac{1}{2}\begin{bmatrix} 2\sigma_v^2 & 0 \\ 0 & 2\sigma_v^2 \end{bmatrix} = \begin{bmatrix} \sigma_v^2 & 0 \\ 0 & \sigma_v^2 \end{bmatrix}.$

For your question, you need to know how to write the density function of each of the $Y_i$'s. Since $Y_i = A + V_i$, the random vector $[\mathfrak{Re}(Y_i),\mathfrak{Im}(Y_i)]$ is a 2D Gaussian vector distributed according to $\mathcal{N}\left(\begin{bmatrix} \mathfrak{Re}(A) \\ \mathfrak{Im}(A) \end{bmatrix},\begin{bmatrix} \sigma_v^2 & 0 \\ 0 & \sigma_v^2 \end{bmatrix}\right)$ i.e. the density function can be written as \begin{eqnarray} \frac{1}{\sigma_v\sqrt{2\pi}}\exp\left(-{\frac{(w-\mathfrak{Re}(A))^2}{2\sigma_v^2}}\right) &\cdot& \frac{1}{\sigma_v\sqrt{2\pi}}\exp\left(-{\frac{(x-\mathfrak{Im}(A))^2}{2\sigma_v^2}}\right)\\& = & \frac{1}{2\pi\sigma_v^2}\exp\left(-\frac{(w-\mathfrak{Re}(A))^2+(x-\mathfrak{Im}(A))^2}{2\sigma_v^2}\right) \\ &=& \frac{1}{2\pi\sigma_v^2} \exp\left(-\frac{(y_i-A)\overline{(y_i-A)}}{2\sigma_v^2}\right) \end{eqnarray} where $y_i=w+jx$ is a "placeholder" for the complex random variable $Y_i$ and the "overbar" denotes complex conjugate i.e. $\bar y=w-jx.$

Finally, we are ready to write the density function of the complex random vector $[Y_1,\ldots, Y_N]$. We note that due to the circular symmetry of each of the components, it is same as the density of the real valued Gaussian random vector $[\mathfrak{Re}(Y_1),\mathfrak{Im}(Y_1),\ldots, \mathfrak{Re}(Y_N),\mathfrak{Im}(Y_N)].$

The likelihood function can now be written as $$p(Y_1,\ldots,Y_N)=\prod_{i=1}^N \frac{1}{2\pi\sigma_v^2} \exp\left(-\frac{(y_i-A)\overline{(y_i-A)}}{2\sigma_v^2}\right)$$ and the negative-log-likelihood becomes \begin{eqnarray} -\log p(Y_1,\ldots,Y_N)&=& N\log(2\pi\sigma_v^2) + \frac{1}{2\sigma_v^2} \sum_{i=1}^N (y_i-A)\overline{(y_i-A)} \\ &=& N\log(2\pi\sigma_v^2) + \frac{1}{2\sigma_v^2} \sum_{i=1}^N (y_i-A)\overline{(y_i-A)} \end{eqnarray}

• Hi, thank you for your answer. Don't you think there should be a minus sign inside the $\exp(.)$ part in the Gaussian pdf (statlect.com/fundamentals-of-statistics/…). Mar 19, 2018 at 3:20
• Good point. Fixed. Mar 19, 2018 at 14:41
• @AtulIngle $\mathbf{Y}$ is not circularly symmetric anymore right? Apr 21 at 16:32

Hi: It's worded more clearly now in that you're estimating A and there's only one RV which makes more sense. Consider the first link I sent in the other message and go to where it says the "likelihood is". Your likelihood is quite similar except, since your variance is multiplied by 2, this causes 2 things to happen in terms of how would you change the likelihood in that document accordingly for your problem.

A) The $2 \pi \sigma^2$ that is getting raised to the $-\frac{n}{2}$ instead becomes $4 \pi \sigma^2_{v}$

B) The denominator of the exp instead of being $2 \sigma^2$ becomes $4 \sigma^2_{v}$.

Make those 2 changes to that likelihood and you will have your expression. I get confused looking at your 2) because my memory of 1/2's and the squares ( in your case H's ) is blurry and it can be confusing. So better to rely on document where likelihood is correct so you can't go wrong. Keep in mind that this is under the assumption that complex RV's don't change much compared to real ones.

• Thank you for your reply. But I don't quite understand why in point A) there should be a 4 since there will not be any sqrt term (please refer the answer to an earlier question dsp.stackexchange.com/questions/40320/…) where if $v \sim CN(0,2\sigma^2)$ then there is no sqrt term.So either it should be only $2\sigma^2$ in the denominator in Eq(3). What do you think? Mar 18, 2018 at 20:12
• the first term in the pdf of the normal is $\frac{1}{2 \pi \times ~ variance}$. The variance is $2 \sigma^2_{v}$. So, you get $2 \times 2$. Mar 19, 2018 at 5:58
• what I said is the density for a single observation. Note that you still have to take the log and turn the product into the sum and all that other fun stuff. but the thing is to have the initial likelihood correct. There have to be tons of examples of computing the log likelihood of the normal on the net, assuming one has the initial likelihood correct and you will have that if you follow what I'm doing. Mar 19, 2018 at 6:01
• Also, I'm not dealing with square roots etc. I'm just telling that the first term you need which corresponds to the first term in the likelihood in the document I sent you in the part 1 has to be $\frac{1}{4 \pi \sigma^2_{v}}$. After that, there may be square roots that need to be taken etc.. Mar 19, 2018 at 6:04