# What is the relation between windowing and hopping in audio DSP

While using short-time Fourier transform (STFT), it is common to use window function, usually tapered.

• Will this not alter the original signal $x[n]$ and the STFT may no more be the true STFT of $x[n]$?
• Also, what is the concept for sliding the frames not by $100\%$ of its width but a fraction of that? i.e, why should frames hop in small steps (for $75\%$ overlap, frames hop by $25\%$). Is there a conceptual explanation with as little math as possible?

What many people do not seen to realize is that using a finite length transform (such as one iteration of an STFT FFT) on a longer signal already alters the longer signal by windowing (e.g. ignoring the portions of the signal that do not fit in the current finite length FFT). Not using a tapered window means that one is using a rectangular window. Using a rectangular window may alter the signal spectrum even more than using a tapered window (from, say the infinite length FT of an infinitely long stationary signal, especially if the signal is not perfectly integer periodic in the STFT FFT aperture width).

The FFT spectrum is altered the least near the center of the window (where a tapered window has a gain of unity or a maximum gain, and non-tapered signal is farthest from both potential edge transients). Therefore the closer together the window centers of individual STFT FFTs (the shorter the hops), the less the alteration or distortion of the total STFT spectrum.

Lots of overlap also provides greater time localization or locality (e.g. the STFT won't miss or slice up a short event that is right between two windows). However the greater the overlap, the greater the computation required. So a compromise overlap of 50% or 75% window overlap is common.

This is gonna be a few steps.

Current discrete time is $n$, hop is $H$, length of window is $L$. fraction of overlap is $\tfrac{L-H}{L}$

if we make

$$\sum\limits_{m=-\infty}^{+\infty} w\left( \tfrac{n-mH}{L} \right) = 1$$

then

\begin{align} x[n] &= x[n] \times 1 \\ &= x[n] \times \sum\limits_{m=-\infty}^{+\infty} w\left( \tfrac{n-mH}{L} \right) \\ &= \sum\limits_{m=-\infty}^{+\infty} x[n] w\left( \tfrac{n-mH}{L} \right) \\ &= \sum\limits_{m=-\infty}^{+\infty} x_m[n-mH] \\ \end{align}

where

\begin{align} x_m[n-mH] & \triangleq x[n] w\left( \tfrac{n-mH}{L} \right) \\ x_m[n] & = x[n+mH] w\left( \tfrac{n}{L} \right) \\ \end{align}

So $x_m[n]$ is a finite-length frame of audio of length $L$ and represents the audio in the neighborhood of $x[n+mH]$ or $m$ hops from the beginning.

Each input frame $x_m[n]$ gets processed into an output frame $y_m[n]$ and the output frames overlap-add to reconstruct the output in the same manner as the input is constructed.

$$y[n] = \sum\limits_{m=-\infty}^{+\infty} y_m[n-mH]$$

So how do you make $\sum\limits_{m=-\infty}^{+\infty} w\left( \tfrac{n-mH}{L} \right) = 1$ ?

(this isn't done yet. i'll get back to this tomorrow.)

Even if you just slide each FFT by 100% of its side, you are indeed, without mentioning it, using a rectangular finite support window, which by itself alone is a form of windowing and, subsequently, alteration.

It has two additional side effects:

• this is a form of block processing, that treats speech frames independently. Which is not so efficient as speech evolves continuously.
• It can produce artifacts, like those observed in JPEG compression, due to the inherent circular fashion of computing the FFT.

With overlapping windows, you can reconcile smooth speech progression, reduced artifacts (clicks, sidelobes, leakage) and simple formulae to allow an accurate reconstruction of the whole information on $x[n]$, provided you use proper windows.

Of course, each frame is somehow altered, but the information pertaining to the samples around the maximum (or center) of the window is faithful enough for standard processing. And all in all, with proper design, you preserve the global information on the whole signal. With a little more math, STFT is an instance complex oversampled filter banks:

in which you can study the conditions under which the analysis stage can retrieve the whole signal from a matching or inverse synthesis filter bank.

Some related questions:

• nice graphic. i am curious of the reason that the taper on the wiindows (they look Hannish or Hammish) goes to zero when the adjacent window is far from the maximum. less than 50% overlap. – robert bristow-johnson May 11 '17 at 3:03
• because $K <2Q$ (or, using my semantic $L<2H$, your window functions are not complementary. you have less than 50% overlap and the little grains or FOFs or whatever you wanna call them, do not add up to the original signal supplied. – robert bristow-johnson May 11 '17 at 15:42
• Got it. I am not a speech processing expert (more the converse). From what I saw, people seem used to 1/4, 1/2 or 3/4 overlaps, for which nice inverses are available. From a pure oversampled filter bank point of view, synthesis is secured under milder conditions than simply complementarity – Laurent Duval May 11 '17 at 18:59