1
$\begingroup$

I am confused as to the difference between neural networks and adaptive filters: As far as I understand it, "neural networks" are largely used for solving inverse problems, where an unknown system is to be identified by the neural network in order to, for example, predict some output. The same is true for linear and nonlinear adaptive filtering algorithms for solving inverse problems such as system identification and predicting the output.

Q: Is there a difference between a linear/nonlinear adaptive filter (chain) that approximates an unknown system and a "neural network" that does the same, or is it called a neural network because it is "fancy"?

$\endgroup$
3
  • 1
    $\begingroup$ you can pretty much define these terms to mean what you want. No, they're not called NNs because it's fancy, but because that naming is useful to these working on things: If you called something you use to filter something adaptively a neural network, you might obscure its use to the filtering community, and conversely, if you called a neural network that has the same structure (by far not all do!) as a nonlinear adaptive filter a filter, you'd be obscuring its nature from the scientists working on neural networks. $\endgroup$ Oct 14, 2021 at 8:57
  • 1
    $\begingroup$ It's very normal that similar things have different names in different fields, so there's nothing (but opinions) to give as an answer here! $\endgroup$ Oct 14, 2021 at 8:57
  • 2
    $\begingroup$ I agree with @MarcusMüller, it is a matter of what you want to communicate. If you're talking about back-propagation, algorithms to do the SGD, neurons, gating functions, etc then saying neural network makes sense. If you're talking about filter taps, filter weights, bandwidth, frequency response, etc then saying adaptive filter makes sense. $\endgroup$
    – Engineer
    Oct 14, 2021 at 9:37

1 Answer 1

2
$\begingroup$

or is it called a neural network because it is "fancy"?

Machine neural networks are called such because they deliberately emulate the functioning of biological neural networks, in an attempt to find ways to solve problems that are not easily solvable by "traditional" signal or data processing means.

Is there a difference between a linear/nonlinear adaptive filter (chain) that approximates an unknown system and a "neural network" that does the same

There is certainly a difference in structure. A feature of both biological and machine neural networks is that they take a large number of inputs (different from a typical "filter"), convolve those inputs with some weighted taps (same as a typical "filter"), then run that result through a nonlinearity to get a "neuron fired, neuron not fired" result (way different from a typical filter).

If you wanted to implement a linear filter with machine neural networks, you'd typically end up with way more "neurons" than your filter would have taps. In biological systems that do this sort of thing (like the more-or-less wavelet processing that happens in your inner ears, before the information gets to the big squishy biological neural network between your ears) "linear filtering" is carried out by either specialized neurons, or other systems (mechanical in case of your ears, electrochemical in case of your eyes, etc.)

Is a neural network an adaptive filter?

That depends on the neural network. A machine neural network, by itself, does not adapt to changing stimuli. The typical machine neural network, as deployed, does not change the weightings or the activation functions -- those are determined during a training phase. Making a neural network that can learn involves adding specific features to the system, and training a neural network is a process that often has unexpected results, so it's not generally something that is allowed to happen unmanaged in the field.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.