This is a continuation of the discussion here. I would comment there, but I don't have 50 rep so I'm asking a new question.

Here's how I understand the DCT step in the MFCC calculation process: The rationale behind it is to separate the correlation in the log-spectral magnitudes (from the filterbank) due to the overlapping of the filters. Essentially, the DCT smooths the spectrum representation given by these log-spectral magnitudes.

Would it be correct to say that the blue line in the image below represents the spectrum as represented by the vector of log-spectral magnitudes, and the red line is that vector once its been DCT-ified?

DCT-ified log-spectral magnitudes (i.e. MFCCs) vs. merely log-spectral magnitudes???

  • $\begingroup$ where can I download your code for testing? $\endgroup$ – auraham Jun 10 '14 at 2:20
  • $\begingroup$ The image below? No image in the post. $\endgroup$ – Eric Platon May 10 '16 at 2:03

Let me start from the beginning. The standard way of calculating cepstrum is following:

$$C(x(t))=\mathcal{F}^{-1}[\log(\mathcal{F}[x(t)])] $$

In case of MFCC coefficients case is a bit different, but still similar.

After pre-emphasis and windowing, you calculate the DFT of your signal and apply the filter bank of the overlapping triangular filters, separated in mel scale (although in some cases linear scale is better than mel):

enter image description here

In respect to cepstrum definition you now represented the envelope of spectrum (reduced spectrum) in mel-frequency scale. If you represent that, then you will see that it kind'a resembles your original signal spectrum.

Next step is to calculate the logarithm of the coefficients obtained above. This is due to fact that cepstrum is supposed to be a homomorphic transformation that separates signal from the impulse response of vocal tract, etc. How?

An original speech signal $s(t)$ is mostly convolved with an impulse response $h(t)$ of the vocal tract:

$$\hat s(t)=s(t)\star h(t)$$

In frequency domain convolution is a multiplication of spectra:

$$\hat S(f) = S(f)\cdot H(f) $$

That can be decomposed into two parts, based on following property: $\log(a\cdot b) = \log(a)+\log(b) $.

We also expect that impulse response is not changing over time, thus it can be easily removed by subtracting the mean. Now you see why we taking the logarithms of our band energies.

The last step in cepstrum definition would be the Inverse Fourier Transform $ \mathcal{F}^{-1}$. The problem is that we have only our log-energies, no phase information, so after applying the ifft we get complex-valued coefficients - not very elegant for all this effort to be a compact representation. Although we can take the Discrete Cosine Transform, which is 'simplified' version of FT and get real-valued coefficients! This procedure can be visualized as matching cosinusoids to our log-energy coefficients. You remember that cepstrum is also called 'spectrum of spectrum'? That's the very step - we are searching for any periodicity in our log-energy envelope coefficients.

enter image description here

So now you see that now it's rather hard to understand how original spectrum looked like. Additionally we are usually taking only first 12 MFCC's, as higher ones are describing fast changes in in log-energies, which is usually making recognition rate worse. So the reasons for doing DCT were following:

  • Originally you must perform IFFT, but it's easier to get the real-valued coefficients from DCT. Additionally we no longer have full spectrum (all frequency bins), but energy coefficients within mel filter-banks, therefore usage of IFFT is a bit of overkill.

  • You see on the first figure that filter banks are overlapping, so the energy from ones next to each other is being spread between two - DCT allows to decorrelate them. Remember that this is a good property for example in case of Gaussian Mixture Models, where you can use diagonal covariance matrices (no correlation between other coefficients), instead of full ones (all coefficients are correlated) - this simplifies things a lot.

  • Other way of decorrelating mel frequency coefficients would be PCA (Principal Component Analysis), technique solely used for this purpose. For our luck it was proven that DCT is very good approximation of PCA when it comes to decorrelating signals, hence another advantage of using Discrete Cosine Transform.

Some literature:

Hyoung-Gook Kim, Nicolas Moreau, Thomas Sikora - MPEG-7 Audio and Beyond: Audio Content Indexing and Retrieval

  • 1
    $\begingroup$ Why can't we take the absolute value of the complex numbers from the IFFT? If the complex numbers are much more difficult to deal with, why do we ever take the IFFT when calculating a cepstrum and not just always do the DCT? Thanks for the explanation. That figure was particularly helpful. $\endgroup$ – acannon828 Apr 30 '14 at 16:48
  • 1
    $\begingroup$ @acannon828: Please see last 3 points of my edited answer. I hope that now it explains everything. $\endgroup$ – jojek Apr 30 '14 at 19:33
  • $\begingroup$ Great response.. Any literature you could attach to this. $\endgroup$ – Bob Burt Apr 20 '17 at 17:42
  • 1
    $\begingroup$ @BobBurt: There you go! $\endgroup$ – jojek Apr 20 '17 at 17:55
  • $\begingroup$ Thanks for the book. Most of the things seem to explained in that one. Does the book also cover the theory about the vocal tract - I doesn't seem to be able to find anything related to that. $\endgroup$ – Bob Burt Apr 20 '17 at 19:24

More than smoothing the DCT reduces the number of dimensions needed to represent the spectrum. DCT is good for dimensionality reduction because it tends to compact most of the energy of the spectrum in the first few coefficients.

  • $\begingroup$ Thanks. This helped me understand what @pichenettes meant by dimensionality reduction. $\endgroup$ – acannon828 Apr 30 '14 at 16:51

The rationale behind it is to separate the correlation in the log-spectral magnitudes (from the filterbank) due to the overlapping of the filters. Essentially, the DCT smooths the spectrum representation given by these log-spectral magnitudes.

This is incorrect. There is correlation between the log-spectral magnitudes not just because they overlap, but also because not any sequence of number represents a "meaningful" (as in, occurring in natural speech and sound) series of log-spectral magnitude. "meaningful" log-spectral magnitudes tend to be rather smooth, with an overall decrease of energy in the higher frequencies, etc. One would say that the dimension of the space of all "meaningful" log-spectral magnitude vectors is smaller than 40 or whichever number of bands you use ; and the DCT can be seen as dimensionality reduction to map the 40-channel data onto this smaller space.

Essentially, the DCT smooths the spectrum representation given by these log-spectral magnitudes.

The DCT does not do any smoothing. You see smoothing when reconstructing from the DCT data - the smoothing being due to the loss of information by the DCT and the coefficient truncation that follows.

But the MFCC coefficients do not store the smoothed spectrum - it stores a sequence of uncorrelated DCT coefficients.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.