0
$\begingroup$

I would like to extend my previous question What is difference between LMS and gradient-descent adaptation? with this other question.

I found out, that RLS and Kalman filter learning seems to be somehow similar. My question is: Can be those algorithms called gradient descent methods? If not, how is this kind of algorithms called?

$\endgroup$
  • 1
    $\begingroup$ Recursive least squares (RLS) filters don't use gradient descent. As their name suggests, they use a least-squares fit to determine the optimum coefficients at each time step. Via clever formulation of the filter structure, one can use the calculations done from time step $n$ to recursively calculate the updated coefficients for time step $n+1$ without having to do the full least-squares fit again. $\endgroup$ – Jason R May 13 '16 at 12:23
  • $\begingroup$ @JasonR Good to know. Does this approach a common name? Can you create an answer from your comment please? $\endgroup$ – matousc May 23 '16 at 9:27
  • $\begingroup$ Done. I'm not sure what you mean about the name of the approach; it's recursive least squares. $\endgroup$ – Jason R May 23 '16 at 12:50
2
$\begingroup$

Recursive least squares (RLS) filters don't use gradient descent. As their name suggests, they use a least-squares fit to determine the optimum coefficients at each time step. Via clever formulation of the filter structure, one can use the calculations done from time step $n$ to recursively calculate the updated coefficients for time step $n+1$ without having to do the full least-squares fit again.

$\endgroup$
  • $\begingroup$ well, the least square solution actually nullify the gradient at each step. $\endgroup$ – LJSilver May 23 '16 at 13:53
  • $\begingroup$ //Via clever formulation of the filter structure,// You mean the lattice structure? $\endgroup$ – Naveen May 23 '16 at 15:08
  • $\begingroup$ Actually, this is far from optimal. And the $\lambda$ dependence turns it a parametric algorithm, such any standard gradient procedure. $\endgroup$ – Brethlosze Jan 16 '17 at 5:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.