EDIT and novel links:
- Deep Learning and Its Applications to Signal and Information Processing, IEEE Signal Processing Magazine, January 2011
- Deep learning: the final frontier for signal processing and time series analysis? "In this article, I want to show several areas where signals or time series are vital"
- I just come back (2018/04/23) from the yearly international conference on acoustics, speech and signal processing, ICASSP 2018. I was amazed by the quantity of papers somewhat relying on deep Learning, Deep Networks, etc. Two pleanaries out of four (by Alex Acero and Yann LeCun) were devoted to such topic. At the same time, most of the researchers I have met were kind of joking about that ("Sorry, my poster is on filter banks, not on Deep Learning", "I am not into that, I have small datasets"), or were wondering about gaining 0.5% on grand challenges, and losing the interested in modeling the physics or statistical priors.
- 2018/01/14: Can A Deep Net See A Cat?, from "abstract cat", to "best cat" inverted, drawn, etc. and somehow surprizing results on sketches
- 2017/11/02: added references to scattering transforms/networks
- 2017/10/21: A Review of Convolutional Neural Networks for Inverse Problems in
Deep learning references "stepping" on standard signal/image processing can be found at the bottom. Michael Elad just wrote Deep, Deep Trouble: Deep Learning’s Impact on Image Processing, Mathematics, and Humanity (SIAM News, 2017/05), excerpt:
Then neural networks suddenly came back, and with a vengeance.
This tribune is of interest, as it shows a shift from traditional "image processing", trying to model/understand the data, to a realm of correctness, without so much insight.
This domain is evolving quite fast. This does not mean it evolves in some intentional or constant direction. Neither right nor wrong. But this morning, I heard the following saying (or is it a joke?):
a bad algorithm with a huge set of data can do better than a smart algorithm with pauce data.
Here was my very short try: deep learning may provide state-of-the-art results, but one does not always understand why, and part of our scientist job remains on explaining why things work, what is the content of a piece of data, etc.
Deep learning requires (huge) well-tagged databases. Any time you do craftwork on single or singular images (i. e. without a huge database behind), especially in places unlikely to yield "free user-based tagged images" (in the complementary set of the set "funny cats playing games and faces"), you can stick to traditional image processing for a while, and for profit. A recent tweet summarizes that:
(lots of) labeled data (with no missing vars) requirement is a deal
breaker (& unnecessary) for many domains
If they are being killed (which I doubt at a short term notice), they are not dead yet. So any skill you acquire in signal processing, image analysis, computer vision will help you in the future. This is for instance discussed in the blog post: Have We Forgotten about Geometry in Computer Vision? by Alex Kendall:
Deep learning has revolutionised computer vision. Today, there are not
many problems where the best performing solution is not based on an
end-to-end deep learning model. In particular, convolutional neural
networks are popular as they tend to work fairly well out of the box.
However, these models are largely big black-boxes. There are a lot of
things we don’t understand about them.
A concrete example can be the following: a couple of very dark (eg surveillance) images from the same location, needing to evaluate if one of them contains a specific change that should be detected, is potentially a matter of traditional image processing, more than Deep Learning (as of today).
On the other side, as successful as DL is on a large scale, it can lead to misclassification of a small sets of data, which might be harmless "in average" for some applications. Two images that just slightly differ to the human eye could be classified differently via DL. Or random images could be set to a specific class. See for instance Deep neural networks are easily fooled: High confidence predictions for unrecognizable images (Nguyen A, Yosinski J, Clune J. Proc. Computer Vision and Pattern Recognition 2015), or Does Deep Learning Have Deep Flaws?, on adversarial negatives:
The network may misclassify an image after the researchers applied a
certain imperceptible perturbation. The perturbations are found by
adjusting the pixel values to maximize the prediction error.
With all due respect to "Deep Learning", think about "mass production responding to a registered, known, mass-validable or expected behaviour" versus "singular piece of craft". None is better (yet) in a single index scale. Both may have to coexist for a while.
However, deep learning pervades many novel areas, as described in references below.
Luckily, some folks are trying to find mathematical rationale behind deep learning, an example of which are scattering networks or transforms proposed by Stéphane Mallat and co-authors, see ENS site for scattering. Harmonic analysis and non-linear operators, Lipschitz functions, translation/rotation invariance, better for the average signal processing person. See for instance Understanding Deep Convolutional Networks.