Framing an audio signal

Question

The first step in MFCC is to split the signal into frames and referring to this discussion on MFCC at this link (Help calculating / understanding the MFCCs: Mel-Frequency Cepstrum Coefficients) @pichenettles said that the frames are usually overlapping. my question is why they are overlapping ?. And if they don't , for example "frame 0 is from 0->440 and frame1 is from 440 ->880 " does this affect the process of feature extraction using MFCC method?

And also why a widowing function must be applied to the frame ?

Thanks.

Answer 1

The MFCC summary you link seems to leave out the typical windowing function applied before each FFT. Segmenting longer data into shorter finite length FFT inputs does an implicit rectangular windowing, which causes the energy of the frequency of any signal not exactly integer periodic in the FFT length to be "spattered" into other distant frequencies of the FFT result. A non-rectangular window function is applied to help reduce these rectangular window artifacts (sometimes called spectral "leakage").

A typical window function, such as Von Hann or Hamming, can be quite lossy at both edges of each window. Overlapping windows adds a window (or more) centered closer to where other windows are lossy, thus helping preserve information about the original signal that would otherwise be lost or degraded.

Another reason is for time resolution. The length of each FFT window might be chosen to give sufficient frequency resolution, however, due to the time-space resolution trade-off of FFT analysis, the resulting time resolution may not be adequate, and thus miss fast audio transients not being centered or isolated in one window. Shorter FFT windows may not provide sufficient frequency resolution. A greater density of FFT windows (using overlap instead of shorter windows) improves the capture and isolation of these shorter time domain events. This greater density (more windows per unit time) can be accomplished by overlapping the windows.

If you don't overlap, then events near or at the boundaries will be severely degraded, and the probability of transients (such a consonants) being between windows, or not being isolated in a single window, is increased, thus reducing how well the MFCC can categorize different inputs.

Answer 2

MFCC is a spectral domain feature. Hence, it is important that a MFCC feature vector carries information solely derived from the sound signal under analysis.

Sound is a non-stationary signal. A spectral domain analysis technique, such as Fourier analysis, is designed to give meaningful interpretation for only stationary signals. Hence, to apply spectral domain analysis techniques to sound, we work with frames. The frame duration is chosen depending on an idea on how fast the spectral content in the signal is varying. In a frame we assume the signal is stationary, and we hence can apply spectral analysis to it. Such a frame analysis is also referred as ``quasi-stationary analysis''. So, framing is a must to use tool when we deal with natural sound signal analysis with methods such as Fourier analysis.

Framing has two aspects: (i) duration, and (ii) window function. The former aspect we already discussed above. On the second aspect, use of window function is a necessary evil. Evil because the frame signal now is multiplied with the window function and hence its Fourier transform will have distortions due to this multiplication. Now, which window function to use? By default, the window used when you just frame the speech is the rectangular window. But rectangular window introduces maximum spectral leakage in the resulting Fourier transform. Alternatives are Hanning window and Hamming window. These have ``less'' spectral leakage because of tapering in time domain. Obviously, as MFCCs are feature of the signal and not of the window, hence you will likely use a tapering window which leads to less spectral distortions.

Now, why overlap? There are at least to reasons for this.

1. Equal weightage to each sample in the sound signal. Owing to the windowing, the end samples in the frame get attenuated. Hence, the MFCC vector for this frame is less affected by these samples. These samples can be encoded in next frame's MFCCs if the next frame has an overlap of at least 50%.

2. Encode redundancy in MFCCs: By overlapping frames (at > 50 %) we have increased redundancy in the MFCCs. However, this redundancy can be exploited in improved delta-MFCCs and delta-delta-MFCCs computation as we will have a more smooth difference function approximation.