Why is each window/frame overlapping?

Question

I am currently reading up on some speech recognition theory. As it speech signals usually are non stationary, but if a speech is seen in a very small time frame can it be deemed stationary, as would only consist of the sound of phoneme.

As explained: http://recognize-speech.com/preprocessing#[object HTMLHeadingElement]

But then this comes up:

As it can be seen in the figure above there can be an overlapping of the windows. The frame length and window length is dependent on the scope of application and algorithms used. In speech processing the value for the frame length typically varies between 5 to 25ms and for the window length between 20 and 25ms [ref.]. Smaller overlapping means larger time shift in the signal, therefore lower processor demand, but the difference of parameter values (e.g. feature vectors) of neighbouring frames can be higher. Whereas larger overlapping can result in a smoother change of the parameter values of the frames, although higher processing power is needed.

My question is how the overlap lower/increase the processor demand?

Answer 1

• Why is each window/frame overlapping?

Windowing is a means to stationarize signals. Inside a small enough window, you can expect that the properties of the signal chunk do not vary too fast. And you now can use tools well-suited to stationary signals, like Fourier-based techniques. You can imagine non-overlapping rectangular windows, each defining a frame. Each sample is, somehow, treated with the same weight (the height of the window). However, when you look at the features extracted from two consecutive frames, or when processing them, the change of property between the frames may induce a discontinuity, or a jump ("the difference of parameter values of neighbouring frames can be higher"). The same phenomenon happens with JPEG images, using $8\times 8$ non-overlapping blocks. This tiling or blocking effect is already annoying for the eye, it is even more disturbing for the ear.

Adding to this, if the signal evolves slowly with time, the left-end or the right-end sample could be seen as the "most different" samples: the frame is inherent localized at a specific location, depending on the window shape. If the window is symmetric, the central location is the center of the frame (some use asymmetric windows for speech processing).

So you can imagine giving some smaller weights to the edges of the frame. This is what you do when you are averaging a signal with unimodal weights, like triangular or Gaussian, instead of uniform. This can be beneficial for frequency estimation, as other windows have better behavior than the uniform one. However, this creates some imbalance in the processing (even information loss as said by @hotpaw2, esp. if the window vanishes) of some samples (the edge ones), unless you compensate it in some way, for instance by choosing a sum of windows whose amplitude ("constant overlap-add") or energy is close to one. Some figures can be found in Spectrum and spectral density estimation by the Discrete Fourier transform (DFT), including a comprehensive list of window functions and some new at-top windows, and the framework of oversampled filter banks provides a quite unified treatment of such constraints.

• My question is how the overlap lower/increase the processor demand?

It depends a lot of the implementation. When the processing is done offline, you can save a lot of redundant operations, by folding the windows into independent blocks, subsampling where you can, etc. Using polyphase matrices or lifting from the overlapped filter bank theory can help you a lot in saving operations. Also, CPU and GPU aware processing can completely change the computational burden deal. So what follows is quite crude.

Without much optimization however, if you have an overlap of $p$%, each sample will be processed, in average, $c = 1/(1-p)$ times. This is about the factor for the number of frames that could be naively processed: $c=2$ for $p=50\%$, $c=4$ for $p=75\%$. If $p= 0$, frames can easily be processed in parallel. Otherwise, some memory or processing is required for the overlap.

At the extreme, you can perform a windowing centered at each sample. This is the principle at the basis of the time-frequency analysis. The computational cost can be high. But the high redundancy has a good side: any processing in the frequency domain would be more robust to noise, or imperfections in the processing: you should hear less clicks and pops.

Answer 2

More overlap means you end up with more windows (of a given length) per second of audio. More windows (of a given length) requires more FFTs which requires more MACs or FLOPs which generally requires more processing power. In return, more window overlap provides greater time locality of information (e.g. on average, random transient events are likely closer to the center of the closest window). However diminishing returns occurs with too much overlap.

Why window (with a tapered shape)? They provide a better frequency response (smoother, less ripples) than the default rectangular windows created by segmenting your audio stream for a sequence of FFTs. The cost is that these windows are lossy (especially near the edges, especially after quantization). Overlapping windows recovers some (or even all) of the lost signal information. (With Von Hann or Hamming windows, try a 50% or 75% overlap to minimize the loss.)