The key concept that you are missing is that you are not just minimising the difference between input and output signals. The error is often calculated from a 2nd input. Just look at the Wikipedia example related to the ECG.
The filter coefficients in this example are recalculated to change the notch frequency of a notch filter according to the frequency extracted from the mains signal. One could use a static notch filter, but you would have to reject a wider range of frequencies to accommodate the variability in the mains frequency. The adaptive filter follows the mains frequency and so the stop band can be much more narrow, thus retaining more of the useful ECG information.
EDIT:
I have looked at this again and I think I understand your question a little better. The LMS algorithm needs an error term in order to update the filter coefficients. In the ECG example that I paraphrase above, I give the error term as a second input from a mains voltage. Now I'm guessing that you are thinking, "Why not just subtract the noise from the signal-plus-noise to leave the signal?" This would work fine in a simple linear system. Even worse, most examples given online tell you (correctly but confusingly) that the error term is calculated from the difference between the desired signal and the output of the adaptive filter. This leaves any reasonable person thinking "If you already have the desired signal, why bother doing any of this!?". This can leave the reader lacking motivation to read and comprehend the mathematical descriptions of adaptive filters. However, the key is in section 18.4 of Digital Signal Processing Handbook, Ed. Vijay K. Madisetti and Douglas B. William.
where:
- x=input signal,
- y=output from filter,
- W=the filter coefficients,
- d=desired output,
- e=error
In practice, the quantity of interest is not always d. Our desire may
be to represent in y a certain component of d that is contained in x,
or it may be to isolate a component of d within the error e that is
not contained in x. Alternatively, we may be solely interested in the
values of the parameters in W and have no concern about x, y, or d
themselves. Practical examples of each of these scenarios are provided
later in this chapter.
There are situations in which d is not available at all times. In such
situations, adaptation typically occurs only when d is available. When
d is unavailable, we typically use our most-recent parameter estimates
to compute y in an attempt to estimate the desired response signal d.
There are real-world situations in which d is never available. In
such cases, one can use additional information about the
characteristics of a “hypothetical” d, such as its predicted
statistical behavior or amplitude characteristics, to form suitable
estimates of d from the signals available to the adaptive filter. Such
methods are collectively called blind adaptation algorithms. The fact
that such schemes even work is a tribute both to the ingenuity of the
developers of the algorithms and to the technological maturity of the
adaptive filtering field
I will keep building on this answer when I get the time, in an attempt to improve the ECG example.
I found this set of lecture notes to be particularly good also: Advanced Signal Processing Adaptive Estimation and Adaptive Filters - Danilo Mandic
