# Learning the Coefficients of Auto Regressive (AR) Model Using Least Mean Squares (LMS) Filter for Signal Prediction

I want to do two things.

1. Estimating Coefficients of AR model using LMS
2. Using Coefficients found in step 1 and predict future samples of a signal using AR equation. I don't have a desired signal so I can't use future samples as well.

The equation of Autoregressive model is $$x(n) = \sum_{i=1}^p a_i x(n-i) + e(n)$$

where $$p$$ indicates the AR model order ($$p=30$$ in my case) and $$\mathbf{a}$$ shows the AR coefficients $$\{a_1, a_2, a_3, a_4,\ldots, a_{30}\}$$.

And LEAST MEANS SQUARE (LMS) algorithm is as follows:

Order=30;      % AR model order
M=30;           % No of filter coefficients same as AR model order
mu=0.001;      % Learning rate/ step size
x=signal(1,1:700);  % length of signal is 700 samples, x= input signal
N=length(x);
Predicted_signal=zeros(1,N);
w=zeros(1,M);               % weights / filter coefficients
for n=M:N
pp=n-M+1;
x1=x(n:-1:pp);
Predicted_signal(n)=(w*x1');
e(n)=x(n)-Predicted_signal(n);     %e(n)=d(n)-y(n); reference signal - actual signal
w=w+2*mu*e(n)*x1;
w1(n-M+1,:)=w(1,:);      % filter coefficients
end

Coefficients (weights/ w) found in previous code to be used in AR equation.

ts = prediction starting point that is 436th sample

Predicted_signal_ar= orignal_signal(1,1:436);
for i=1:Order
t=ts-i;
prediction1(1,i)=(w(1,i)*Predicted_signal_ar(t)); %% putting lms coefficients in AR equation
end

s1(1,a)=sum(prediction1(1,:));
Predicted_signal_ar(1,ts)=s1(1,a);

There is some error in my code and I am not sure what is it. The above code should give me close to actual Coefficients which then I pass to step 2 for AR prediction. In results of 2nd step there is a sudden jump in prediction of first few points and prediction performance is also poor.

• I cant say anything about the code but it's very unlikely that you need to go as high as 30 lags in your AR model. I don't know matlab but there are model selection criteria for deciding how many lags are "pseudo-optimal". Box- Jenkins- Reinsel is not my favorite for arima modelling but it's generally viewed as the bible and will atleast point you to good references for model selection. Chatfield is another. Ledolter also comes to mind. There are so many. Jul 8, 2019 at 3:46
• I have selected and fixed the model order using Akaike Information criterion (AIC) so practically there is no problem in model selection. Jul 8, 2019 at 4:20
• ok. gotcha. Model selection is not easy. maybe it's different in DSP, but atleast in econometrics, you would never see an AR model with 30 lags. These authors are supposed to be AIC masters so check this out if you're interested. guianaplants.stir.ac.uk/seminar/materials/… Jul 8, 2019 at 14:12
• What I got from the code is that you are using x1 as the input of your filter in the LMS, which is a vector extracted from x. However, when you compute the error, you compare the output to the x(n) in the same time instant. Should you not compare it to a future sample? Otherwise, you are not predicting anything, you are only trying to find the filter taps that, given the input samples from your input signal, which the most recent sample is in instant $n$, to predict the signal at instant $n$, i. e., you will end up finding an impulse as impulse response of your filter, not the actual AR model. Jul 8, 2019 at 18:51
• @Abeeha, Could you share the signals with us?
– Royi
May 19, 2021 at 16:54

In order to use the LMS to learn an AR Model one should use the predictor variant of the Least Mean Squares (LMS) filter.

Basically we predict the $$x \left[ n \right]$$ sample using past samples: $$\left\{ x \left[ n - i \right] \right\}_{i = 1}^{k}$$ where $$k$$ is the LMS filter order.

This will basically give us the AR Model of any signal we'll drive into the LMS filter.

I don't have you samples so I created a similar signal using a linear combination of 30 sine signals with different amplitudes, frequencies and phases:

I used 75% of the signal for the LMS training (Blue) and the rest is unseen by the LMS.

The prediction is:

Though the input signal isn't generated by an AR Model, using a model of order 101 the results seems to be pretty good.
results might get even better when the model order is tweaked better.

The code is available at my StackExchange Codes Signal Processing Q59325 GitHub Repository (Look at the SignalProcessing\Q59325 folder).