The context is as follows: I want to measure (as in, digitize) some input signal $x$ (I have no detailed knowledge about $x$ or its statistical description).

A noise signal $x_N$ contaminates $x$ before I digitize it. That is, the signal I read is: $x' = x + H_1(x_N)$, where $H_1$ is some unknown filter (let's assume linear, but possibly time-varying).

I have access to $x_N$ (for example, $x_N$ could be the "mains" voltage, or the ripple voltage in my power supply). I want to cancel the noise component in $x'$ to obtain the best estimate of $x$. This estimate $\hat{x}$ is obtained by subtracting a filtered version of the noise: $$\hat{x} ~=~ x' - \;H(x_N)$$

where $H$ is my adaptive filter. The best estimate is that with minimum power (as a function of the filter coefficients), since the noise and the true signal are uncorrelated.

My algorithm aims to minimize $\left(\hat{x}\right)^2$. At time step $n$, I have: $$\left(\hat{x}(n)\right)^2 = {\left( x' - \sum_{k=0}^L h_k \, x_N(n-k) \right)}^2$$ To obtain the gradient, I take: $$\frac{\partial \left(\hat{x}(n)^2\right)}{\partial h_k} = -2 \, \hat{x}(n)\; x_N(n-k)$$ And presumably voilà: each $h_k$ at step $n$ is updated by adding $\mu$ times the above expression to the estimate of $h_k$ at step $n-1$.

Question 1: Does the above make sense?

Question 2: Rather, a concern when I look at the math above --- intuitively, I expect the gradient's magnitude to approach zero as the system gets close to the solution (the optimal filter), but that doesn't seem the case from the equation above: the gradient is a copy of the last $L$ samples of $x_N$ (which in principle has non-zero mean), scaled by the present value of the output (which also, in principle has non-zero mean).

Am I doing something wrong?


1 Answer 1


I'm a little skeptical of your derivation and notation as it abstains from telling whether a Mean-Square or a Least-Squares metric of error is minimized, but rather just works on an instantaneous error at sample $n$. But I believe you eventually would devise an LMS or RLS filter, hence either of the two would happen.

So, based on your notation the adaptive system error is $\hat{x}(n)$, which is an estimate of the true signal $x(n)$, and you have two correlated noises $x_N$ and $H(x_N)$ which are uncorrelated with the true signal. The standard noise cancelling setup.

Had you used the steepest descend based quadratic MSE surface minimization procedure, your instantaneous gradient on the error surface would be something like:

$$ \nabla J(w) = - E\{ e(n) \bar{x}[n] \} $$

where the expectation is due to stochastic MSE framework. In your notation this would become:

$$ \nabla J(w) = - E\{ \hat{x}(n) \bar{x}_N[n] \} $$

Now, if the optimum condition is reached, then the error is minimized, and we can assume that $\hat{x}(n)$ converged to the true signal $x(n)$ and you may replace it. Remember this is an approximate conditions actually...

$$ \nabla J(w) = - E\{ x(n) \bar{x}_N[n] \} $$

and since your noise source is uncrorrelated with the true signal, this expectation is zero, yielding the zero gradient at the minimum point.

In your case, you don't specifically use expectations, so might be harder to show why the gradient would go to zero. Furthermore, as always, be aware that gradient would only theoretically go to zero. Any practical implementation will deviate from this depending on how unrealistic your assumptions were.

  • $\begingroup$ I'm still a bit at a loss; I want to implement this (in C/C++), so I would need a solid grasp of every little detail. I may not be familiar with the standard notations; is $J$ the standard notation for cost function, and is $w$ the vector of filter coefficients? Also, what exactly is the function that we minimize? I assume we're talking about LMS type of algorithm (MSE is the tool for optimum filters, right? a fixed filter obtained if we have apriori knowledge of the statistical description of the signals, right?). What is the concrete expression for that expectation? $\endgroup$
    – Cal-linux
    Commented Jan 8, 2019 at 14:37
  • $\begingroup$ oh, also: what is $\bar{x}_N$ ? $\endgroup$
    – Cal-linux
    Commented Jan 8, 2019 at 14:53
  • $\begingroup$ A short answer is hard to provide. There is no offical standard for these things ;-)) Yet it's better using similar notation from some well known publications on adaptive filters. That $bar{x}_N$ is the block of input noise samples into the FIR tapped delay line filter structure. $\endgroup$
    – Fat32
    Commented Jan 8, 2019 at 19:55
  • $\begingroup$ @fat32 could you name some of the "well known publication"? $\endgroup$ Commented Feb 7, 2019 at 6:45
  • 1
    $\begingroup$ @user6522399 begin with Simon Haykin, Widrow, Stearn, Sayed, books on Adaptive Filter Theory, Adaptive Systems etc and check all the end of chapter citations which includes introductory references to all the well known publications from IEEE electrical engineering domain... where well known refers to. $\endgroup$
    – Fat32
    Commented Feb 7, 2019 at 10:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.