0
$\begingroup$

I am learning estimation methods following the book of Steven Kay, "Fundamentals of Statistical Processing, Volume I: Estimation Theory "

Theory says that if the measurement noise is Gaussian, the OLS = MLE. However, in the noise-free case, why am I not getting the true estimates? I have not checked for the noisy case. Can somebody please help in correcting my code and the concept as to why none of the estimation methods such as the yule-walker, OLS and MLE are working to give the true estimates of an AR(2) model. Where am I going wrong?

clear
h =[1,0.195,- 0.95]; %true coeffficients
L=length(h);
N=256;
 x=round(randn(1,N)); %driving signal
 
y  = filter(1,h,x); % generate the AR(2) model
arcoeffs_yule = aryule(y ,2)
arcoeff_mle = mle(y)
Y=y';
x=x';
arcoeffs_ols =  [Y -[0;Y(1:end-1)] -[0;x(1:end-1)]]\x


arcoeffs_yule =

    1.0000    0.9285    0.0000


arcoeff_mle =

   1.0e+07 *

   -0.0941    7.8185

arcoeffs_ols =

    0.1405
   -0.1513
    0.1527

UPDATE to the code: based on the comment, I have put the length of the AR model=3 inside aryule instead of passing the order 2. Then I have passed some initial values to the filtered signal. Yet getting incorrect estimates.

   clear
    h =[1,0.195,- 0.95]; %true coeffficients
    L=length(h);
    N=256;
     x=round(randn(1,N)); %driving signal
     
y(1) = 0.1;
y(2) = 0.2;

% Generate the AR model.
for i =3 : N
    y(i) = 0.195 *y(i-1) -0.95*y(i-2) + x(i);
end

%y = filter(1,h,x);

arcoeffs_yule = aryule(y,L)
arcoeffs_mle = mle(y)
Y=y';
x=x';
arcoeffs_ols =  [Y -[0;Y(1:end-1)] -[0;x(1:end-1)]]\x

arcoeffs_yule =

    1.0000   -0.0420    0.9202    0.1261
    
arcoeffs_mle =

    0.0040    2.9161

arcoeffs_ols =
  0.1108    0.0263    0.1613
Improve this question
$\endgroup$
17
  • $\begingroup$ HI: I'm not familiar with matlab but A) you have a 1 in h but, if it is an AR(2) that you are generating, then $h$ should be of length two. B) your filter call is using $x$ which are the $\epsilon_t$ observations. You need to filter with some starting values for $x$ ( you need two because it's AR(2) ) and then call filter on those observations rather than on $\epsilon_t$. You can set the initial values to whatever you want because, for a long enough series, their effect will wear off. $\endgroup$
    – mark leeds
    Aug 10, 2020 at 20:03
  • $\begingroup$ Based on your suggestions, I have updated the code. Still I am getting incorrect estimates. $\endgroup$
    – Sm1
    Aug 10, 2020 at 23:22
  • $\begingroup$ A) I'd have to look at the code more carefully because they shouldn't be that different but why would you have 2 AR MLE coefficients but three for OLS ?. If it's an AR(2) without intercept, you should have 2. Also, you should be getting similar but not identical estimates because the relation only holds when the underlying DGP has normal error term and is linear in x ( the standard regression model ). The AR(1) has lagged dependent variables on its RHS so OLS estimation, is, technically, not correct to even use for estimation. But. you can often still get quite reasonable estimates. $\endgroup$
    – mark leeds
    Aug 11, 2020 at 1:55
  • $\begingroup$ Hi: When you generate the simulated data, that looks correct. But I don't understand how mle or aryule functions can come back with coefficient estimates if you don't send them the underlying model ? In other words, how do they know that it's an AR(2) ? The ols coefficient code is totally greek to me unfortunately. Not clear on the \x at the end. $\endgroup$
    – mark leeds
    Aug 11, 2020 at 2:07
  • $\begingroup$ @markleeds: Thank you, your help in making the code workable would be extremely helpful to me. It is indeed strange that MLE gives 2 coefficients. I am not sure about the theory on normal error term.I plotted the histogram of y which did was not bell shaped. $\endgroup$
    – Sm1
    Aug 11, 2020 at 2:08

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.