MLE formulation -- confusion regarding the terms in the equation (Part1)

Question

If $v \sim CN(0,2\sigma^2_v)$ is a circularly complex Gaussian random variable which acts as the measurement noise in this model $$y_n = x_n + v_n \tag{1} $$ where $x \sim CN(0,2\sigma^2)$, then is the following log-likelihood expression correct?

$$P_x(x_1,x_2,...,x_N) = \prod_{n=1}^N\frac{1}{2\pi \sigma^2_v} \exp \bigg(\frac{-{({y_n-x_n})}^H ({y_n-x_n})}{2\sigma^2_v} \bigg) \tag{2}$$

Confusion:

I am having a doubt whether there will be a 2 in the denominator in (3) from the term $2\sigma^2_v$ or not. Am I doing it correctly? Please help.

Answer 1

The answer is no, your likelihood is not correct because $\mathbf{x}$ is a random variable. Equation 2 should be: $$ p( \mathbf{y} \mid \mathbf{x} ) \quad \text{not}\quad p(\mathbf{x}) $$ You have a prior distribution on $\mathbf{x}$. MLE is suitable for "deterministic but unknown" $\mathbf{x}$. Also, you observe $\mathbf{y}$ not $\mathbf{x}$ (or at least that is implied by the use of $\sigma_v$).

You can approach your problem using conjugate priors (Bayesian, as Mark Leeds suggests)

https://en.wikipedia.org/wiki/Conjugate_prior

or as a MAP estimate

https://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation

which is better is application dependent. You need to choose. The MAP handles range constraints on unknowns nicely, it is often a hybrid MLE.

The Normal or Gaussian distribution has a conjugate distribution (although knowing or not knowing the variance on the prior is different) so you have a posterior pdf as your estimate, at which point you can take a mean, or a max, or a median as a point estimate.

With the advent of Monte Carlo Markov Chain (MCMC) and programs like BUGS and Stan, the Bayesian approach is gaining ground.

https://en.wikipedia.org/wiki/OpenBUGS

https://en.wikipedia.org/wiki/Stan_(software)

One of the best things about being a Signal Processor, is that often, different sets of assumptions will produce acceptable algorithms, but now you can trade off complexity versus throughput, hardware, battery life, cost ....

Statisticians tend to be more particular about how assumptions relate to bias.

Answer 2

Hi: I don't have time to carefully look at everything right now and I'm unfamiliar with circular complex random variables. But, if it's similar to regular normal rv's and the two variables are independent , then their sum has mean zero and their variance is $ 2 \sigma^2 + 2 \sigma^2_{v}$. So, the likelihood can be written having mean zero and that variance. But it's not clear to me what you're estimating. There aren't any coefficients in the model ( implied one's I guess somewhere or maybe you are doing bayesian ? ) so what is actually observed ? $x_{n}$ or $v_{n}$. That has to be known if you want to write the likelihood correctly. If $x$ is viewed as fixed and known and not a random variable, then what you wrote is missing a factor of 2 in the factor before the exp. The $2 \pi$ has to be multiplied by the variance which is $2 \sigma^2_{v}$. Also, it's not clear to me how X can be viewed as fixed since you said in the question that it was an RV also. If it is an RV and also not observed, then that has to be accounted for in the likelihood. That's all I can tell you and even what I'm saying may not be correct if complex means something totally different. Good luck.