1
$\begingroup$

I am learning estimation theory and need help in understanding for educational purpose how the concept of ML works with the help of a step by step implementation. I am trying to find out the ML estimates of the coefficients of a moving average model. I am unable to understand what I need to do in order to get the log-likelihood curve, so that I can select the coefficients based on the curve. There are implementations in the internet but I wanted to make my own in order to get a clear picture. The way I have done is for the model

 y(i) =  0.6*x(i-1) ;
 z(i) = y(i) + w(i)

where the true coefficient is h = [ 0.6], the input is x and w is zero mean additive white gaussian noise with variance 1. I have used 4 different values of the estimates out of which one value is the true. If the implementation is correct, then I should get estimated coefficient as 0.6 because I am using the true known coefficient in the estimation procedure. However, this is not so. Where am I going wrong?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

The maximum likelihood estimator will give you precisely the true value for the parameter for a vanishingly small number of instances --- unless the noise is zero.

Your noise instance w is not zero-mean, which will probably give more of a bias away from the true value.

If I take your numbers are just change the last entry in w from -0.65 to 0.1 then I get the "right" answer. Note that this changes the mean of w form -0.16 to -0.042 (i.e. closer to zero).

Your values:

enter image description here

The new values:

enter image description here

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you for the details. I corrected my implementation and have got correct values till $(h*x-z)*(h*x-z)$. But, the values of the likelihood in the last column are very different from mine as a result I am getting the ML estimate of h as 0.4692. $\endgroup$
    – SKM
    Commented Nov 12, 2016 at 0:41
  • $\begingroup$ Could you please let me know what is the formula that you used to calculate the likelihood values? I am getting different values and I checked using L(iter) = exp(-sum((z(k)-new_output(iter,k)).^2)/(2*sigma2w)); still the values are different. $\endgroup$
    – SKM
    Commented Nov 12, 2016 at 0:41
  • $\begingroup$ Lastly, I had used the randn(N,1) command to generate noise samples. I think in Matlab, we can generate zero mean noise samples using this command. How can I generate noise samples which are zero mean like the ones that you suggest. Thank you very much once again $\endgroup$
    – SKM
    Commented Nov 12, 2016 at 0:42
  • $\begingroup$ @SKM I didn't use the full equation. Just the sum of the squared term. $\endgroup$
    – Peter K. ♦
    Commented Nov 12, 2016 at 0:42
  • $\begingroup$ I dont' quite understand this part. Would it be possible to include that in your answer please how you calculated the likelihood values using the the sum part ? Appreciate your help very much $\endgroup$
    – SKM
    Commented Nov 12, 2016 at 0:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.