I have been studying the adaptive filters lately and now when I am at RLS (Recursive Least Squar) Algorithm I came across the term used in the weighting function of the RLS called forgetting factor ($\lambda$). This term ‘memoryless’ itself confuses me. My doubt is:

  • How is memoryless RLS different than the standard RLS? Does it involve some specific value for lambda (as in, it is said that if $\lambda$ $= 1$, system has infinite memory).

I would be glad if anyone could help.


In its bare classical form the RLS algorithm, recursively (for every new iteration), solves the classical problem of least squares; by computing the optimal FIR transversal filter coefficients $w[n]$ which minimizes the sum of error squares between the produced filter output $y[n]$ and a desired response $d[n]$ where the error sum is based on the entire data set beginning from the first sample $y[0]$ to the last sample (the current sample) $y[n]$. Therefore as time progresses, $n \to \infty$, the range of data set also goes to infinity, which means that the memory of the algorithm is infinite; i.e., the current value of the output $y[n]$ and the FIR coefficients $w[n]$ depend (indirectly via recursion) on the entire sample set.

A consequence of the infinite memory is that the estmation becomes more and more exact and refined given that the inputs were WSS random process. On the other hand a clear drawback of this infinite memory dependency is the fact that if the inputs are not WSS then the filter response to reflect the new conditions is slow (the entire past which is irrelevant to the current computation still effects the output due to infinite memory) and tracking is lost.

In order to improve this last situation, and therefore to make the algorithm track non stationary inputs, its memory should be shortened; i.e., only a finite amount of last local samples should effect the current computations. This is achieved by a forgetting factor $\lambda$ embedded into the recursion equations. The range of $\lambda$ is between $0$ and $1$ where $\lambda = 1$ converts the algorithm to classical form with infinite memory with no tracking and $\lambda \to 0$ has less and less memory and more responsive with better tracking...

  • $\begingroup$ Hmm that makes sense. I was confused in the term 'memory-less' which now if I understood correctly doesnt mean 'memory-less' but 'less-memory' because as $\lambda$ can't be zero (because of its presence in the denominator of the $\Psi^{-1}$) we can not get rid of it. Also, I think that we can make $\lambda$ even smaller and it with the past values of the coefficients fades away quickly. $\endgroup$
    – Copernicus
    Dec 14 '17 at 18:31
  • $\begingroup$ Yes; the smaller the forgetting factor $\lambda$ the faster the past values will fade away, hence the shorter will be the memory of the filter... Note that the term memory does not refer to the physical storage space (as registers of a digital filter etc) but the effect of past values on the present. $\endgroup$
    – Fat32
    Dec 14 '17 at 18:40
  • $\begingroup$ Yes, I get it now. Thanks a lot @Fat32. $\endgroup$
    – Copernicus
    Dec 14 '17 at 19:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.