Are least square filters, or filters that minimize error energy, the same as least mean square adaptive filters?


2 Answers 2


TL;DR: No, they are not necessarily the same.

Gory Details

Least squares is just an optimization technique. It is used in a variety of ways.

For filter design it is used to select that realizable filter $H_r(e^{j\omega})$ that most closely matches, in the least squares sense, the ideal required filter response $H_i(e^{j\omega})$: $$ H_r(e^{j\omega}) = \arg \min \parallel H_r - H_i \parallel_2 $$ where $\parallel \cdot \parallel_2$ is the 2-norm or least-squares norm.

This sort of filter $H_r$ is not adaptive. That is, it doesn't change once it has been designed.

Adaptive filters may also use the least squares criterion, but in a different way: as part of the adaptation step.

Adaptive filters start off with initial filter coefficients $\vec{w}_o[0]$ and then use an update: $$ \vec{w}_o[n] =\vec{w}_o[n-1] + \mu g[n-1] $$ where $\mu$ is the step-size and $g$ is the gradient of the least squared error surface in the direction of the minimum (from our current "location" of $\vec{w}_o[n-1]$).

Here, $g$ is determined by our error criterion: least squares. This means: $$ \parallel \vec{w}_{\tt opt} - \vec{w}_o \parallel_2 $$ where $ \vec{w}_{\tt opt}$ is the unknown optimal (minimizing) solution.


In complement to Peter, yes or no, depending on what you want to be adaptive to. Let us assume the world is made of signals AND systems. Signals pass (by temporary existence), while systems remain (by essence). I acknowledge that one may swap these philosophical terms, on instance.

In most lectures, systems are somehow considered invariant, while signals can change a lot, often according to some ordinal variable (time, space, temperature, etc.). In this context, most methods adapt modeled systems according to a given target, under some assumptions. Any "least" method is a form of adaptivity, yet with respect to some "invariant" template.

The common notion of adaptivity is generally thought with respect to a variation of the data along the ordinal variable, where the actual model is, within a given formalism (spectral, auto-regressive, moving average, etc.), adapted to the actual properties of a chunk of the signal.

However, the actual situation is more blurry than that. However, from a corpus point of view, all least square filters are not adaptive, as the existence, or the dependence on some ordinal variable is not mandatory.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.