I just simulated an auto-regressive second-order model fueled by white noise and estimated the parameters with normalized least-mean-square filters of orders 1-4.
As the first-order filter under-models the system, of course the estimations are weird. The second-order filter finds good estimates, although it has a couple of sharp jumps. This is to be expected from the nature of NLMS filters.
What confuses me is the third- and fourth-order filters. They seem to eliminate the sharp jumps, as seen in the figure below. I can't see what they would add, as the second-order filter is enough to model the system. The redundant parameters hover around $0$ anyway.
Could someone explain this phenomenon for me, qualitatively? What causes it, and is it desirable?
I used step size $\mu=0.01$, $10^4$ samples, and the AR model $x(t)=e(t)-0.9x(t-1)-0.2x(t-2)$ where $e(t)$ is white noise with variance 1.
The MATLAB code, for reference:
% ar_nlms.m function th=ar_nlms(y,order,mu) N=length(y); th=zeros(order,N); % estimated parameters for t=na+1:N phi = -y( t-1:-1:t-na, : ); residue = phi*( y(t)-phi'*th(:,t-1) ); th(:,t) = th(:,t-1) + (mu/(phi'*phi+eps)) * residue; end % main.m y = filter( , [1 0.9 0.2], randn(1,10000) )'; plot( ar_nlms( y, 2, 0.01 )' );