I have been doing some readings on the computation of Mel-Frequency Cepstral Coefficients (MFCC) and further use of Vector Quantizers (VQ) for recognition purposes. I am however stumped by the method of computation for those MFCCs regarding the number of suggested frequency bands. For example, why is the recommended number of mel filterbank filters 20 -40? Why do we eventually end up with 13 coefficients and not 40 as will have been found from the 40 log energies we shall calculate?
Number of filter banks
One of the last steps in the MFCC's calculation is measuring the energy in the filter banks. We do that because want to reduce the dimensionality of our input vector (amplitude spectrum), as well as capture its envelope. Those triangular filters are spaced over the Mel scale:
This means that we have very good resolution in low frequencies. Exactly opposite is true for higher frequencies. We do that because MFCC's are suited for speech-related tasks and most of the information is located in lower frequencies (i.e. formants).
So how many filter banks do we actually want? For example HTK by default is using 20 filter banks. You might increase that number, especially if you are dealing with signals that contain a lot of closely spaced frequencies and you want to resolve them between each other. It's totally up to you - in the end what really matters is the classification performance.
Number of coefficients
After taking the logarithm of energies in each filter bank, the last step (or second last if you are doing some liftering) is to calculate the Mel Frequency Cepstral Coefficients. We do that by fitting the cosines to calculated log energies using the DCT. This captures the periodicity in the reduced spectrum. Figure below should help you in understanding that process. You can imagine that for number of coefficients equal to number of filter banks this would correspond to capturing the alternating energy between each of the filter banks.
So how many MFCC's do we want to calculate? HTK by default uses 12 and in most applications it is more than enough. In general we don't want too many coefficients because:
- It's all about reducing the dimensionality of our feature space.
- One DCT properties is that it de-correlates and keeps most of the information in first few coefficients.
Again it all depends on your application and you should adjust this number based on the recognition performance. From my experience the gain from increasing the number of MFCC's was negligible compared to introducing the $\Delta$ and $\Delta\Delta$ coefficients.
A great answer is given above, but to specifically respond to why 12 MFCC per frame. Usually, when you are doing automatic speech recognition, you do not care about the pitch information of the signal, that is, the information of the vocal cords. Rather, you are more interested in the phoneme-related information or the information due to the shape of the vocal tract that creates the sounds.
Pitch value usually has a low period and when you move to the cepstrum domain (using DCT or IDFT) that is captured in the mid to end values. This is why most applications only chose the first 12 or 13 MFCCs so as to not include the pitch information. If you are building a system where the pitch information is important, you should increase the number of MFCCs (maybe you want to perform speaker identification)