I am currently working on extracting features for speech recognition purposes. I wanted to try an approach similar to MFCC features in which the center frequency of each band pass filters are placed a distance of x mel displaced of each other.

mel distant placed band pass filters

But instead of looking at the audio in 1d I see it in 2d using the spectrogram as representation, and I was thinking of using a CNN instead of using band pass filters , with different weight for sets of frequencies, as they are interpreted differently, and some more distinct than others.

Illustration of spectrogram of an audio

Elaborate on some are more distinct than others:

Consider the normal mel scale. Frequency between 0 - 1000 are interpreted linearly increasing, and above as logarithmic increasing. The way I interpret this is that the ear is more susceptible to differences in the lower range (<1000), and not so much in the higher range (>1000). So in my head it makes sense to place the weight as higher in the lower range, and decreasing the higher range.

Plot of the frequency vs mel

But what about the kernel size and the center frequency?.. As I enter the logarithmic range, will the band pass filters used in the MFCC become wider, which in this case should result in larger kernels, but what about the center frequency?.. Where or how should i place a kernel such that a term such as center frequency would make sense related to kernels?... is it even possible, or am I just being silly?


1 Answer 1


CNNs sort of irrelevant here because they are applicable when you have some invariant transform - shift or rotation. The idea of CNN is that weights are shared for different data points, not that weights are different. You are thinking about simple DNN.

CNNs are also applied in speech recognition, but for a bit different task. For example, you can consider multiple frames jointly and tie filter shape for different frequencies. This way system learns Gabor-like filters. See Robust CNN-based Speech Recognition With Gabor Filter Kernels.

You can try to train your own kernels from raw data, they will eventually resemble bandpass filters. See Acoustic Modeling with Deep Neural Networks Using Raw Time Signal for LVCSR. This gives a marginal improvement.

You can also start from default bandpass filters as initial values and see if network learns an improved version.

You can also restrict network architecture to consider only the region around central frequency and just learn the weights in that region. Such network will unlikely improve anything.

  • $\begingroup$ i guess you misunderstood.. Yes CNN generally have FWS but, I divide the spectrogram into section and apply a specific neural network to each section. Interesting paper about gabor, maybe usefull with my approach microsoft.com/en-us/research/wp-content/uploads/2016/02/… $\endgroup$
    – Bob Burt
    Mar 5, 2017 at 22:17
  • $\begingroup$ The key point for you to understand is Figure 3 in Microsoft paper and the label "Share same weights". Note the word "share". $\endgroup$ Mar 5, 2017 at 22:30
  • $\begingroup$ Just to be sure... I have not misunderstood it right.. I divide the spectrograms into sections and apply CNN on that section.. That way the weight is only shared in that section and nothing else. $\endgroup$
    – Bob Burt
    Mar 6, 2017 at 12:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.