1
$\begingroup$

The early stages of the Convolutional Neural Networks are performing classical convolutions with a certain kernel size on the input image. Is is possible to express in common terms the type of convolutions they are generally performing (lowpass, highpass, differential, anisotropic...), or are they completely application-dependent/impossible to interpret ?

Alternatively, can you show some sample kernel ? I would be very grateful.

Improve this question
$\endgroup$
4
  • $\begingroup$ well, the "shape" of these kernels depend on the lets-call-it-features they need to detect in the input. For example, think about object recognition in images. If you need to find cars in photos, you don't care about the grain and shot noise in cameras, so it's fair to assume there's a low-pass manifestating somehow. How that happens exactly is up to the network itself. On the other hand, your algorithm will probably not care for very large-scale parts in your pictures (street surfaces from afar), so there will probably be also some high-passy behaviour. $\endgroup$
    – Marcus Müller
    Nov 3, 2017 at 9:55
  • $\begingroup$ @MarcusMüller: do you have concrete data from existing cases ? $\endgroup$
    – user7657
    Nov 3, 2017 at 9:58
  • $\begingroup$ no, sorry, I don't. $\endgroup$
    – Marcus Müller
    Nov 3, 2017 at 10:04
  • $\begingroup$ Take a look at the visualizations in the link below, this might give some idea: adeshpande3.github.io/adeshpande3.github.io/… $\endgroup$
    – T3am5hark
    Mar 30, 2018 at 6:36

1 Answer 1

Reset to default
2
$\begingroup$

The kernels used by a ConvNet are nothing but neural weights. You can think of them as a multilayer perceptron with some connections cut off and some weights restricted to be equal (weight sharing).

With this in mind, we must take into account that the kernels (or filters, in this context) are learned, so they depend exclusively on the type of inputs and the desired outputs. Each learning process is unique for a set of inputs/outputs, and so there is no way to know a priori what the filters will turn out to be. They are, as you state in your question, application-dependant.

They can be interpreted, though. Take for example a simple case, where we are trying to classify images that consist of horizontal lines only, and others of vertical lines only. If you do this, then the kernels will indeed look like that: vertical and horizontal lines. That's because the filters have learned common patterns to look for on the inputs, and they have acquired that form in order to be able to match those patterns and maximize their correlation to them.

In Figure 3 at Krizhevsky et al., there are some kernels for you to see. Kernels looking for horizontal lines will consist of horizontal lines. Kernels looking for vertical lines will consist of vertical lines. And so on.

To see this more clearly, you can train a CNN and take one of the kernels learned and filter any image with it to see what happens.

I think that a paragraph from this website can be of relevance:

So what can we conclude from these feature maps? It's clear there is spatial structure here beyond what we'd expect at random: many of the features have clear sub-regions of light and dark. That shows our network really is learning things related to the spatial structure. However, beyond that, it's difficult to see what these feature detectors are learning. Certainly, we're not learning (say) the Gabor filters which have been used in many traditional approaches to image recognition. In fact, there's now a lot of work on better understanding the features learnt by convolutional networks. If you're interested in following up on that work, I suggest starting with the paper Visualizing and Understanding Convolutional Networks by Matthew Zeiler and Rob Fergus (2013).

In the paper mentioned in the previous quote, there are lots of examples of learned kernels, related to the images that were used to train the network. I think that paper can give you some useful insight.

Improve this answer
$\endgroup$
11
  • $\begingroup$ IMO, the convolution coefficients are more than neural weights as they take into account the relative spatial arrangement of the pixels. The kernels shown in the paper (thanks for this reference) are striking: the top ones seem to be higher order derivatives filters in a few directions. They remind of Gabor filters. Those in the bottom are more first order derivatives (derivative of Gaussian) but in a certain direction of color space. $\endgroup$
    – user7657
    Nov 3, 2017 at 14:14
  • $\begingroup$ They are neural weights, but the fact that they are forced to accomplish weight sharing (which is the same as taking into account the spatial arrangement) are the thing that makes the CNNs be so intuitive (as you can see the weights as a convolution, which we are so used to). Nevertheless, they still depend on the application, as they may vary a lot depending on what features the network has learned to be important. Some of them will always appear, but some of them will be unique for certain features. It will also depend on the number of kernels and layers you set (i.e. the hyperparameters). $\endgroup$
    – Tendero
    Nov 3, 2017 at 14:49
  • 1
    $\begingroup$ these examples are actually hard to believe, unless they were chosen from a filter bank of edge detectors: they are so perfectly straight ! $\endgroup$
    – user7657
    Nov 3, 2017 at 15:33
  • 1
    $\begingroup$ @endolith That seems to be the case with the SuperVision(?) incarnation of AlexNet, see Fig. 6 of the paper. The network also learned "various colored blobs" (authors' words). But not all later networks trained on images seem to have high-frequency Gabor wavelets in the first layer, more like just the first order or two. $\endgroup$
    – Olli Niemitalo
    Nov 3, 2017 at 18:56
  • 1
    $\begingroup$ "Receptive field" is the biological equivalent of a kernel. The average receptive field of neurons in cortical columns in an animal called tree shrew look very much like Gabor wavelets, see Extended Data Figure 6 of Lee et al. (2016) Topology of ON and OFF inputs in visual cortex enables an invariant columnar architecture. $\endgroup$
    – Olli Niemitalo
    Nov 3, 2017 at 19:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.