0
$\begingroup$

After applying STFT to a signal, you typically need to convert the frequencies to the Mel scale. Typically the frequency values range from $0$ to $N/2$, where $N$ is the width of each sliding window.

I have read so many articles about Mel Spectrogram. Basically they all say to convert frequencies to Mel Scale in the following way:

  1. Partition the Hz scale (i.e. partition a sequence of numbers ranging from $0$ to $N/2$). In this way we will have a number of bins. This number is for example indicated in librosa by "n_mel". These bins are called frequency bands.

  2. Covert each bins into Mel Scale using the formula $2595*log(1+f/700).$

  3. Application of triangular filters for each bins to capture the energy at each frequency band and approximate the spectrum shape.

It seems that when this is buried in the articles, nobody really knows why these operations are carried out. I find it difficult to fully understand:

  1. Why divide the Hz scale into bins?

  2. What exactly is meant by a frequency band?

  3. Why do we apply triangular filters?

  4. What exactly is a triangular filter? How is it defined mathematically and from what concept?

Can anyone explain exactly these things to me or recommend a book where these steps are explained in detail? There seems to be no book that explains these concepts exactly.

$\endgroup$
0
$\begingroup$

This was my first google hit: https://medium.com/analytics-vidhya/understanding-the-mel-spectrogram-fca2afa2ce53

It seems to explain both how and why?

Mel frequency binning is afaik only used when human perception is involved. Either directly, or for signals that have probably been adapted to human hearing (such as human speech). It is a way to represent signal information in a way that mimics «low level auditory perception», or could be seen as removing perceptually redundant information.

I have not understood why it is that we generally have the same temporal reolution in narrow low frequency bands, as in wide high frequency bands. Historically, it may have been due to limited processing power and the efficiency of a single STFT windowing process. And/or it could reflect the limited neuron firing rate after the acoustic-mechanical filtering carried out by our ear.

But from a dsp perspective, I would expect a variable time/frequency resolution, where high & wide bands gets more temporal resolution.

$\endgroup$
1
  • $\begingroup$ Thank you very much for your reply. I also read your article, very well written and made me understand the spectogram mel better. However she doesn't answer the 4 questions I asked. $\endgroup$ – Francesco Ladogana May 20 at 22:41
0
$\begingroup$

The below tutorial has the clearest explanation I have read about Mel, I think it can answer all of your questions.

Mel Frequency Cepstral Coefficient(MFCC) tutorial

$\endgroup$
1
  • $\begingroup$ I have already read this article and learned a lot. But still it remains ambiguous, it does not fully answer my questions. $\endgroup$ – Francesco Ladogana Jun 14 at 15:35
0
$\begingroup$

As I am a newbie in DSP and I thought this article has a clear explanation, I did not try to answer your questions myself. But you said you have read it and thought these questions are still ambiguous, then I want to try one more time to give my answers.

First, about the concepts of bins and bands, they confused me a lot in the beginning when I was trying to figure out what the Fourier Transform means, especially the Discrete Fourier Transform (DFT), which is also much related to your questions. Right now my understanding of frequency bins are specific frequency values, such as 2000, or 2001.5. A frequency band, for example from 1000 to 2000, covers a range of frequencies, and it is defined by the two frequency bins 1000 and 2000 (one is the floor, and one is the ceiling). In the physical world, the values of frequencies can take any real number, in other words, they are continuous. As to the above example, there are infinite values or frequency bins in the frequency band [1000,2000]. However, when comes to digital signal processing, in the time domain, we take finite (and integer) numbers of samples with a sampling rate, for example 44100 samples per second (i.e. 44100Hz). Later we do Discrete Fourier Transform (DFT) in ways of FFT or STFT to go to the frequency domain. When we do STFT, we do it in the unit of frame and apply hanning window to it. As you mentioned the width of the window equals to the frame size such as 2048 samples, and 2048 is your N.

As to your questions:

  1. It is not that we further partition a frequency scale into bins, naturally we get N/2 frequency bins after FFT or STFT. If N = 2048, sampling_rate = 44100, then we have 1024 frequency bins [21.53, 43.07, 64.60, ..., 22028.47, 22050]. Our finite and discrete sampling in the time domain has already limited the frequencies we get to be finite and discrete too, as bins.

  2. As explained above, usually we filter(maybe can be understood as combine or group) the frequency bins to get bands.

  3. Naturally we have more than one way to group bins to bands, I think the triangular filter is just one specific (Mel) way to get Mel spectrogram.

  4. As to this question, the above tutorial definitely explains better than me.

About the question 3, I am not sure on what level you are asking about the reason to do the triangular filter. If it is about the choice of the specific technique/algorithm, I think it is related to the question 4, and I will leave it to the tutorial. If it is about the bigger reason to do Mel transform, this comes to the field of human sensation and perception. When we talk about frequency and amplitude of the sound source,these are pure physical features of sound. Pitch and loudness are their corresponding human perceptual features. The point is our human perceptions do not linearly change with the values of the physical features. That is the reason we have Decibel scale and Mel scale, therefore our perception change linearly with the values in these scales. We only need to do some math transforms to find the real physical values corresponding to these scales or vice versa.

I hope these can offer some help.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.