# Confusion with regards to STFT and MFCCs

I'm wondering if someone can clear the following up for me as I'm having a difficult time understanding the concept.

I'm looking to implement and learn about the processes and uses using short-time Fourier transforms. I have had experience using DFT's and wanted to expand my knowledge.

From research that I have read online, as well as in some text books I understand that the STFT is calculated by the following:

1. Take the inputted signal of $x[n]$

2. Multiply $x[n]$ by a window function (e.g. Hamming, Hanning, Triangular, .., Barlett) by sliding the block size over the time axis

3. Compute the DFT for the resulting values of step 2

Is this correct? I know the following steps above are for usually a continuous-time STFT and in the case of a Discrete-time STFT the data can be broken up into chunks.

The other thing that is confusing me:

For a project last year, I had to implement a MFCC algorithm. Inside this algorithm I computed the "Mel Scale" in Triangular filter bank and then multiplied each of these values against the FFT (DFT) computed on the inputted signal in Discrete-Time. Is this therefore a use of the STFT, just using the Mel-Frame over Hamming etc.. Or, would I still need to multiply the result of the FFT (DFT) with the Triangular filter bank if I was to use a STFT in order to calculate the MFCC values?

• You missed 2 steps. Preemphasis (between 1 & 2) and zero padding (between 2 & 3). – user13107 Aug 15 '13 at 1:54
• @user13107 But surly zero padding will only happen if the window size is less than the FFT window? Also, I don't know why pre-emphasis would need to be applied here? – Phorce Aug 15 '13 at 11:47
• Correct. But for the sake of completeness it needs to be mentioned. Preemphasis is usually applied to speech signals. It is done to nullify the effect of spectral rolloff and effectively flatten the spectrum. If you are not using speech data, then it's not needed. – user13107 Aug 15 '13 at 14:12
• @user13107 Thank you for your feedback :) I will allow for zero-padding and for pre-emphasis I don't know why I didn't mention this above. mhm! – Phorce Aug 15 '13 at 16:46

I am going to break down this answer into two main parts, one for a step-by-step computation of the STFT, and the other in regards to how to compute the MFCC's, once you have the STFT.

STFT: Let us say you wanted to compute the STFT of your signal $x[n]$, of total length $N=100$, and block size $M = 10$, with $50$% overlap, and using a hamming window. I will give you a step by step, (and redundant) flow, to get you started:

1. Take the first $M=10$ samples out, that is, samples $x, x, x ... x$.
2. Window them, by multiplying them, element by element, with your window coefficients.
3. Take the DFT of this result.
4. Take the next $M=10$ samples out, that is, samples $x, x, x, ... x$.
5. Window them, by multiplying them, element by element, with your window coefficients.
6. Take the DFT of this result.

etc, etc. You get the idea. Let us say that you took a (same size) DFT in the above example. Then, you will end up with a $10$ x $19$ complex matrix. If you then simply take the absolute magnitude of the above matrix, you will end up with a real matrix, corresponding to the absolute magnitude of your complex matrix. This is called the Short-Time-Fourier-Transform, or STFT. We will use the STFT as the input into computation of the MFCCs.

Before that, let us get some formulas and terminology straight: Let us call our STFT matrix $S$. This matrix is composed of $B=19$ blocks, or columns, and $K=10$ frequency bands, or rows. To refer to a time-frequency point in $S$, we will say $S[k,b]$, where $b \in 1, 2, ... B$, temporal blocks, (columns) and $k \in 1, 2, ... K$ sub-bands, (rows).

MFCC:

Once you have the STFT computed, you can go ahead and use that as a stepping stone for computing the MFCC's. Regarding your question, you seem to be confusing the time domain windowing done on each block for the STFT, and the frequency domain windowing done in the MFCC. They are completely separate and unrelated.

Let us say that you computed your STFT using a hamming window, and you have a $K=500$ x $B=1000$ real matrix. Thus using our prior terminology, you have a real matrix $S_{K \text x B}$. Thus, you have $500$ sub-bands, (frequencies), and $1000$ blocks, (time).

Let $Y_{b}[k] = |S[k,b]|^2$. That is, we are letting $Y_{b}[k]$ represent the power-spectral density of a particular column/block of the STFT matrix $S$. Obviously, each $Y_{b}[k]$ is a $500$ x $1$ column vector.

Clearly, we have $1000$ $Y$'s. That is, we have $Y_{1}[k], Y_{2}[k], Y_{3}[k], ... Y_{1000}[k]$.

Now, you want to compute the MFCC of each column. Take the first column. This is $Y_{1}[k]$. Forget about all the other columns, and forget about the hamming window. The first column is now used in your MFCC algorithm. The first step in MFCC was getting a PSD column, which you have, because we have $Y_{1}[k]$.

The second step is the computation of, say, 10 power estimates. This is computed by windowing this column, by 10 overlapping windows, as per the mel-scale, like so: Note how each (triangular) window is mostly zeros. So now you literally do a dot product of each window, with $Y_{1}[k]$. You will get $10$ numbers, for this column. Now take the log of those $10$ numbers. Now take the DCT of this result. Congratulations! You just computed the MFCC for this column. You just computed the MFCCs for $Y_{1}[k]$.

Now, go back, and repeat for all the other $Y_{2}[k]$. Then repeat for $Y_{3}[k]$, etc.

• It's that simple? It seems simple enough from that explanation. BUT can you use multiple Windowing functions? For example, Hanning/Hamming over triangular window? NOTE: The Triangular window is not what I multiplied last time, instead, I multiplied against the Mel Scale.. Would I still need to multiply the STFT results with the Melscale? – Phorce Aug 14 '13 at 16:28
• @user1582478 Yes, it is that simple. You can use any window you like. Hamming, Hann, Triangular, Martian, whatever. You seem to be confused about MFCC. MFCC is not a window. MFCC is an algorithm. The input into the MFCC algorithm is a block, like $x, x, ... x$. The output is a series of Mel-Frequency-Cepstral-Coefficients. You can run the MFCC algorithm on each column of the absolute magnitude squared of the complex STFT. – Tarin Ziyaee Aug 14 '13 at 16:39
• Thank you for your reply. I know the MFCC is an algorithm. I just did not think the Mel Scale was a windowing function (like Hamming etc..) I'm looking back at my old project and I do the following: result[i] += mel[i][j] = value[j+melstart[i]] and for value I did the following: value[i] = fabs(FFT[i].re + FFT[fft_size - i - 1].re + FFT[i].im - FFT[fft_size - i - 1].im)/2; I'm just really confused as if to say this: Once I have obtained my STFT results by using a Hamming window.. Do I just forget about the Mel Scale? – Phorce Aug 14 '13 at 16:47
• @user1582478 Please see my edits. – Tarin Ziyaee Aug 14 '13 at 17:13
• Amazing answer :) The edit has really cleared it up! I was using an STFT before, I just didn't realise it ha! Now, I'm going to code this up :) Thank you very much for your time, much appreciated! – Phorce Aug 14 '13 at 17:23