In this paper stands:

The derivation and analysis of NLMS rest upon the usual independence assumptions.

It has a footnote:

The independence assumptions used in the analysis of adaptive filters are:

  1. sequences $x(k)$ and $w(k)$ are zero mean, stationary, jointly normal, and with finite moments
  2. the successive increments of tap weights are independent of one another; and
  3. the error and $x(k)$ sequences are statistically independent of one another

I have several problems with that:

  1. Who and when choose this assumptions? Is it something general or it just appears in the first publication on this topic?

  2. I think in practice it is not possible to work just with stationary inputs $x(k)$. And also input and the adaptive weights also will never be zero mean (especially during online usage when it is impossible to normalize the input data). Also the error and input are strongly correlated, so how it can be statistically independent?

  3. Is there some other theory dealing with analyzing those more real-life applications where those assumptions cannot be fulfiled?

If is it possible to answer, than please answer. If my questions are too stupid, please correct me. Thanks in advance.

  • 1
    $\begingroup$ what is "$w[k]$"? i know the basics of the LMS and NLMS, but the signal semantics i remember are $x[n]$ for the FIR input, $y[n]$ for the FIR output, $d[n]$ for the desired signal (compared to $y[n]$), and $e[n] \triangleq y[n] - d[n]$ as the "error" signal. $\endgroup$ Commented Aug 21, 2016 at 0:27
  • 1
    $\begingroup$ $w(k)$ are the adaptive weights. So $y(k) = w(k)x(k)$ $\endgroup$
    – matousc
    Commented Aug 21, 2016 at 0:40
  • $\begingroup$ okay, cool. i would call the weights the same as the FIR coefficients. and your math is wrong. should be: $$ y[n] = \sum\limits_{k=0}^{L-1} w[k] x[n-k] $$ $\endgroup$ Commented Aug 21, 2016 at 1:17
  • $\begingroup$ yea, I miss the transposition and does not properly markup the vectors, so $y(k) = \textbf{w}(k)^T \textbf{x}(k)$, but this is not point of my questions (and $k$ stands for discrete time index, you probably use $n$ for this) $\endgroup$
    – matousc
    Commented Aug 21, 2016 at 1:26
  • $\begingroup$ ok all I would say is this: if you take it strictly mathematical, then no signal in our universe will be stationary (or exactly independent). These assumptions are actually kind of approximations just as Newtonian mechanics is an approximation under suitable conditions. So you must be careful enough to distinguish between proper and improper conditions under which these assumption will make sense. You can follow more advanced approaches by putting less assumptions on the signals but that would increase the computational cost. ok I admit Newton's laws are far better approximations ! $\endgroup$
    – Fat32
    Commented Aug 21, 2016 at 15:06

1 Answer 1


I think there is an error in your referenced independence assumption. $w(k)$ should be the update part $\Delta w(k)$ i.e the

$w(k+1)=w(k)+\mu \Delta w(k) =w(k)+\mu \frac{x(k)*e(k)}{c+x(k)^Hx(k)}$

The error after convergence is uncorelated and zero mean.

The two other assumptions are correct.

If your filter is fast enough you may use it at nonstationary systems as long as you are able to track the variations.

  • $\begingroup$ Sorry, but If I understand correctly, you mean that "\Delta \textbf{w}(k)" should be in first assumption instead of "\textbf{w}(k)"? If yes, why? $\endgroup$
    – matousc
    Commented Aug 23, 2016 at 1:18
  • $\begingroup$ If all w(k) had zero mean then the filter would be pointless, the delta w(k) is the update/regression vector and it will after convergence have zero mean. $\endgroup$ Commented Aug 23, 2016 at 5:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.