1
$\begingroup$

If $v \sim CN(0,2\sigma^2_v)$ is a circularly complex Gaussian random variable which acts as the measurement noise in this model $$y_n = x_n + v_n \tag{1} $$ where $x \sim CN(0,2\sigma^2)$, then is the following log-likelihood expression correct?

$$P_x(x_1,x_2,...,x_N) = \prod_{n=1}^N\frac{1}{2\pi \sigma^2_v} \exp \bigg(\frac{-{({y_n-x_n})}^H ({y_n-x_n})}{2\sigma^2_v} \bigg) \tag{2}$$

Confusion:

I am having a doubt whether there will be a 2 in the denominator in (3) from the term $2\sigma^2_v$ or not. Am I doing it correctly? Please help.

$\endgroup$

2 Answers 2

3
$\begingroup$

The answer is no, your likelihood is not correct because $\mathbf{x}$ is a random variable. Equation 2 should be: $$ p( \mathbf{y} \mid \mathbf{x} ) \quad \text{not}\quad p(\mathbf{x}) $$ You have a prior distribution on $\mathbf{x}$. MLE is suitable for "deterministic but unknown" $\mathbf{x}$. Also, you observe $\mathbf{y}$ not $\mathbf{x}$ (or at least that is implied by the use of $\sigma_v$).

You can approach your problem using conjugate priors (Bayesian, as Mark Leeds suggests)

https://en.wikipedia.org/wiki/Conjugate_prior

or as a MAP estimate

https://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation

which is better is application dependent. You need to choose. The MAP handles range constraints on unknowns nicely, it is often a hybrid MLE.

The Normal or Gaussian distribution has a conjugate distribution (although knowing or not knowing the variance on the prior is different) so you have a posterior pdf as your estimate, at which point you can take a mean, or a max, or a median as a point estimate.

With the advent of Monte Carlo Markov Chain (MCMC) and programs like BUGS and Stan, the Bayesian approach is gaining ground.

https://en.wikipedia.org/wiki/OpenBUGS

https://en.wikipedia.org/wiki/Stan_(software)

One of the best things about being a Signal Processor, is that often, different sets of assumptions will produce acceptable algorithms, but now you can trade off complexity versus throughput, hardware, battery life, cost ....

Statisticians tend to be more particular about how assumptions relate to bias.

$\endgroup$
2
  • 1
    $\begingroup$ I answered your original question. This is a new question. I would prefer that you accept it and post your new question, or ignore this answer. I prefer not to engage in an endless loop. $\endgroup$
    – user28715
    Mar 18, 2018 at 8:24
  • $\begingroup$ I have posted a new question dsp.stackexchange.com/questions/47933/… Can you please take a look, if you can help. Thanks $\endgroup$
    – Ria George
    Mar 18, 2018 at 18:31
3
$\begingroup$

Hi: I don't have time to carefully look at everything right now and I'm unfamiliar with circular complex random variables. But, if it's similar to regular normal rv's and the two variables are independent , then their sum has mean zero and their variance is $ 2 \sigma^2 + 2 \sigma^2_{v}$. So, the likelihood can be written having mean zero and that variance. But it's not clear to me what you're estimating. There aren't any coefficients in the model ( implied one's I guess somewhere or maybe you are doing bayesian ? ) so what is actually observed ? $x_{n}$ or $v_{n}$. That has to be known if you want to write the likelihood correctly. If $x$ is viewed as fixed and known and not a random variable, then what you wrote is missing a factor of 2 in the factor before the exp. The $2 \pi$ has to be multiplied by the variance which is $2 \sigma^2_{v}$. Also, it's not clear to me how X can be viewed as fixed since you said in the question that it was an RV also. If it is an RV and also not observed, then that has to be accounted for in the likelihood. That's all I can tell you and even what I'm saying may not be correct if complex means something totally different. Good luck.

$\endgroup$
4
  • $\begingroup$ Hi: 'm only referring to 2) and not the rest of it. The likelihood for the normal is here. statlect.com/fundamentals-of-statistics/… Notice how the $\sigma^2$ multiplying the $2\pi$ on is the same as the $\sigma^2$ right after the exp. In your case, I don't know what the variance is ( is it the sum of the 2 rv's as I said earlier or just the one rv ? Whatever it is, it has to be the same in both places. I don't want to comment on this further because you have 2 variances and I don't know what you're estimating. $\endgroup$
    – mark leeds
    Mar 18, 2018 at 16:06
  • $\begingroup$ The CRLB is a bound for the variance of an estimator. For example, $\bar{x}$ would be an estimator for $\mu$. In your case, I'm pretty confused so I don't want to confuse you more. The only thing I can be sure of is what I said about the likelihood given that there's only one RV. According to Stanley, there are 2 RV's so I should not say more. Figure out what's being estimated and that will help a lot for understanding what to do. $\endgroup$
    – mark leeds
    Mar 18, 2018 at 16:09
  • $\begingroup$ Hi: Below is a link to what, at a glance, seems like a nice explanation of CRLB with EE type examples. cs.tut.fi/~hehu/SSP/lecture2.pdf. As I said, the first step is to figure out what's being estimated and what you're estimator is. then at that point, you can deal with calculating the correct likelihood and the derivatives etc. good luck. $\endgroup$
    – mark leeds
    Mar 18, 2018 at 16:35
  • $\begingroup$ Thank you for your answer and for providing the links. They are quite useful. BUt it is very difficult to do for the complex variable. Based on your comments, I have posted a new question here dsp.stackexchange.com/questions/47933/… $\endgroup$
    – Ria George
    Mar 18, 2018 at 18:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.