When studying neural networks from Neural Networks and Learning Machines, by Simon Haykin, the author highlights the close similarity between of adaptive filtering and neural networks.

From a scalar-valued signal, if we put a tapped delay input along with an ADALINE (Adaptive Linear Neuron), do we have an adaptive filter? From this point of view, both areas become closely related.

PS: A good reference is MathWorks - Adaptive Neural Network Filters


1 Answer 1


I think adaptive filter, and a single layer perceptron with MSE error, will be equivalent.

  • $\begingroup$ The main problem imho is that, for adaptive filters, $\mathbf{x}[n] = (x[n], x[n-1], ...)$. It is not necessarily true for neural networks. That is where a tapped delay line comes in. $\endgroup$ Commented May 6, 2023 at 15:54
  • $\begingroup$ Moreover, if we are talking about linear adaptive filtering, a nonlinear activation function prevents they from being equal. That is where ADALINE comes in. $\endgroup$ Commented May 6, 2023 at 15:57
  • $\begingroup$ A single layer perceptron with MSE loss is exactly like adaptive filter as I wrote. You align the samples in batches to match the input. $\endgroup$
    – David
    Commented May 6, 2023 at 18:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.