0
$\begingroup$

This question is based on the application of the pdf which was an earlier question of mine asked here Confusion regarding pdf of circularly symmetric complex gaussian rv

If $v \sim CN(0,2\sigma^2_v)$ is a circularly complex Gaussian random variable which acts as the measurement noise in this model $$y_n = A + v_n \tag{1} $$ where $y$ is the observation and $A$ is a scalar unknown value which needs to be estimated. I am having a slight confusion whether there will be a 2 in the denominator of Eq(3) and Eq(4) with the $\exp(.)$ term. Based on the answer in the link, there should be no sqrt term with $\pi$ in the denominator, if $v \sim CN(0,2\sigma^2_v)$. If $v \sim N(0,\sigma^2_v)$ then there is a sqrt term.

Can somebody please check if I have correctly written out the log-likelihood? I think I am missing a 2 in the denominator of $\exp[.]$ term in Eq(3) but I am not quite sure.

Thank you for your time and help.

$$P_y(y_1,y_2,...,y_N) = \prod_{n=1}^N\frac{1}{2\pi \sigma^2_v} \exp \bigg(\frac{-{({y_n-A})}^H ({y_n-A})}{2\sigma^2_v} \bigg) \tag{2}$$

taking log $$\ell = -N\ln(2\pi\sigma^2_v) - \frac{1}{\sigma^2_v} {\bigg[{[\sum_{n=1}^{N} {(y_n - A)}{(y_n - A)}^{\mathsf{H}} ]}\bigg]}. \tag{3}$$ $$ = -N\ln(2\pi\sigma^2_v)- \frac{1}{2 \sigma^2_v}{\bigg[ \sum_{n=1}^{N}y_n y_n^\mathsf{H} - 2 \sum_{n=1}^N y_n A\bigg]} - \frac{1}{2 \sigma^2_v}{\bigg[ \sum_{n=1}^N {AA}^\mathsf{H} ] \bigg]} \tag{4}$$

$\endgroup$
  • $\begingroup$ Going from (2) to (3), why did $(y-A)^H(y-A)$ turn into $(y-A)(y-A)^H$? $\endgroup$ – Atul Ingle Mar 19 '18 at 1:54
  • $\begingroup$ @AtulIngle: I used the commutative law in multiplication: A.B = B.A $\endgroup$ – Ria George Mar 19 '18 at 2:04
  • 1
    $\begingroup$ Ah, so $(\cdot)^H$ really means complex conjugate here because these are complex scalars. In general, this is not true for vectors/matrices. $\endgroup$ – Atul Ingle Mar 19 '18 at 2:05
2
$\begingroup$

I looked this up on Wikipedia: A complex Gaussian random variable $V = \mathfrak{Re}(V)+j\mathfrak{Im}(V)$ is said to be zero mean circularly symmetric $\mathcal{CN}(0,\Gamma)$ if the random vector $[\mathfrak{Re}(V),\mathfrak{Im}(V)]$ is a Gaussian random vector with mean $[0,0]$ and covariance matrix $ \frac{1}{2}\begin{bmatrix} \mathfrak{Re}(\Gamma) & -\mathfrak{Im}(\Gamma) \\ \mathfrak{Im}(\Gamma) & \mathfrak{Re}(\Gamma) \end{bmatrix} = \frac{1}{2}\begin{bmatrix} 2\sigma_v^2 & 0 \\ 0 & 2\sigma_v^2 \end{bmatrix} = \begin{bmatrix} \sigma_v^2 & 0 \\ 0 & \sigma_v^2 \end{bmatrix}. $

For your question, you need to know how to write the density function of each of the $Y_i$'s. Since $Y_i = A + V_i$, the random vector $[\mathfrak{Re}(Y_i),\mathfrak{Im}(Y_i)]$ is a 2D Gaussian vector distributed according to $\mathcal{N}\left(\begin{bmatrix} \mathfrak{Re}(A) \\ \mathfrak{Im}(A) \end{bmatrix},\begin{bmatrix} \sigma_v^2 & 0 \\ 0 & \sigma_v^2 \end{bmatrix}\right)$ i.e. the density function can be written as \begin{eqnarray} \frac{1}{\sigma_v\sqrt{2\pi}}\exp\left(-{\frac{(w-\mathfrak{Re}(A))^2}{2\sigma_v^2}}\right) &\cdot& \frac{1}{\sigma_v\sqrt{2\pi}}\exp\left(-{\frac{(x-\mathfrak{Im}(A))^2}{2\sigma_v^2}}\right)\\& = & \frac{1}{2\pi\sigma_v^2}\exp\left(-\frac{(w-\mathfrak{Re}(A))^2+(x-\mathfrak{Im}(A))^2}{2\sigma_v^2}\right) \\ &=& \frac{1}{2\pi\sigma_v^2} \exp\left(-\frac{(y_i-A)\overline{(y_i-A)}}{2\sigma_v^2}\right) \end{eqnarray} where $y_i=w+jx$ is a "placeholder" for the complex random variable $Y_i$ and the "overbar" denotes complex conjugate i.e. $\bar y=w-jx.$

Finally, we are ready to write the density function of the complex random vector $[Y_1,\ldots, Y_N]$. We note that due to the circular symmetry of each of the components, it is same as the density of the real valued Gaussian random vector $[\mathfrak{Re}(Y_1),\mathfrak{Im}(Y_1),\ldots, \mathfrak{Re}(Y_N),\mathfrak{Im}(Y_N)].$

The likelihood function can now be written as $$ p(Y_1,\ldots,Y_N)=\prod_{i=1}^N \frac{1}{2\pi\sigma_v^2} \exp\left(-\frac{(y_i-A)\overline{(y_i-A)}}{2\sigma_v^2}\right) $$ and the negative-log-likelihood becomes \begin{eqnarray} -\log p(Y_1,\ldots,Y_N)&=& N\log(2\pi\sigma_v^2) + \frac{1}{2\sigma_v^2} \sum_{i=1}^N (y_i-A)\overline{(y_i-A)} \\ &=& N\log(2\pi\sigma_v^2) + \frac{1}{2\sigma_v^2} \sum_{i=1}^N (y_i-A)\overline{(y_i-A)} \end{eqnarray}

$\endgroup$
2
$\begingroup$

Hi: It's worded more clearly now in that you're estimating A and there's only one RV which makes more sense. Consider the first link I sent in the other message and go to where it says the "likelihood is". Your likelihood is quite similar except, since your variance is multiplied by 2, this causes 2 things to happen in terms of how would you change the likelihood in that document accordingly for your problem.

A) The $2 \pi \sigma^2$ that is getting raised to the $-\frac{n}{2}$ instead becomes $4 \pi \sigma^2_{v}$

B) The denominator of the exp instead of being $2 \sigma^2$ becomes $4 \sigma^2_{v}$.

Make those 2 changes to that likelihood and you will have your expression. I get confused looking at your 2) because my memory of 1/2's and the squares ( in your case H's ) is blurry and it can be confusing. So better to rely on document where likelihood is correct so you can't go wrong. Keep in mind that this is under the assumption that complex RV's don't change much compared to real ones.

$\endgroup$
  • $\begingroup$ Thank you for your reply. But I don't quite understand why in point A) there should be a 4 since there will not be any sqrt term (please refer the answer to an earlier question dsp.stackexchange.com/questions/40320/…) where if $v \sim CN(0,2\sigma^2)$ then there is no sqrt term.So either it should be only $2\sigma^2$ in the denominator in Eq(3). What do you think? $\endgroup$ – Ria George Mar 18 '18 at 20:12
  • $\begingroup$ the first term in the pdf of the normal is $\frac{1}{2 \pi \times ~ variance}$. The variance is $2 \sigma^2_{v}$. So, you get $ 2 \times 2$. $\endgroup$ – mark leeds Mar 19 '18 at 5:58
  • $\begingroup$ what I said is the density for a single observation. Note that you still have to take the log and turn the product into the sum and all that other fun stuff. but the thing is to have the initial likelihood correct. There have to be tons of examples of computing the log likelihood of the normal on the net, assuming one has the initial likelihood correct and you will have that if you follow what I'm doing. $\endgroup$ – mark leeds Mar 19 '18 at 6:01
  • $\begingroup$ Also, I'm not dealing with square roots etc. I'm just telling that the first term you need which corresponds to the first term in the likelihood in the document I sent you in the part 1 has to be $\frac{1}{4 \pi \sigma^2_{v}}$. After that, there may be square roots that need to be taken etc.. $\endgroup$ – mark leeds Mar 19 '18 at 6:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.