I have wrote this algorithm for steepest descent:
function [e2,w_opt]=SteepestDescent_Alg2(x,d,itr,w) N=length(w); sample=length(x); x=[zeros(N-1,1); x];%padding x1=zeros(sample,N); for i=1:N x1(:,i)=x(N-(i-1):end-(i-1)); end R=x1'*x1/sample; P=x1'*d/sample; lambda=eig(R); mu=1/(min(lambda)+max(lambda)); w_opt=zeros(size(w)); dd=(d.'*d)/(sample); e2=zeros(1,itr); for i=1:itr grad=2*R*w_opt-2*P; e2(i)=dd+w_opt'*R*w_opt-2*P'*w_opt; w_opt=w_opt-mu*grad; end wc=R\P end
These lines forming R:
x1=zeros(sample,N); for i=1:N x1(:,i)=x(N-(i-1):end-(i-1)); end R=x1'*x1/sample;
But if we use other way to calculating R:
function R=corlnm2(h,N) %This function obtains the correlation matrix of the output %of an FIR filter which is derived by a white noise, and its %coefficients are given in vector "h". %"N" specifies the dimensions of "R". % % R=corlnm2(h,N) %j=sqrt(-1); a=size(h); if a(1)>a(2) h=h'; end m=length(h); for k=1:N if k<=m R(k)=h(k:m)*h(1:m-k+1)'; else R(k)=0; end end R=toeplitz(R); end
The code have written for this problem, also assume there is coloring filter with tap numerator coefficient of
Both two way of estimating R, getting value, really close together, but the second way gives weird learning curve.
Also the input noise set to 0.1 variance, thus minimum kesi(Mean Square Error) must be 0.1, calculating R in first manner give correct learning cure(plot of MSE vs itteration) but the
W's are extremely close to
even with more than 0.001 precision, why? And I think both way must give same answer why but this isn't ...?