0
$\begingroup$

I am attempting to overlap-add time-domain audio frames in a real-time context (with a continuous stream of data coming-in). Each frames consist of 1024 samples and have been previously "Hann" windowed in view of performing FFT/IFFT operations.

When the hopsize is set to exactly 50% (here, 512 samples), everything works fine as the frames "concatenate" in a way that it creates a flat-top. Consequently, there isn't any amplitude build-up occurring: enter image description here

However, as soon as I try to set the overlap to a greater value, let's say 75% (=768 samples), the frames expectedly stack-up and their respective amplitudes accumulate in a troublesome way: enter image description here

Unfortunately, I couldn't find any clear documentation on how to properly normalize overlapping frames based off the hopsize value, the framesize and the window type.

What would be the proper way of scaling my audio signal ?

Thanks in advance

$\endgroup$
  • 2
    $\begingroup$ Why the downvote ? I'm fairly new here, so if something isn't clear in my question or totally out of topic, I'd really appreciate more insights. $\endgroup$ – Darius Petermann Nov 16 '18 at 6:52
  • $\begingroup$ I didn't downvote but your question title says hopsize > 50% and the question says hopsize < 50%. That could be clarified. Also, overlap-add is known as a method of FFT--accelerated convolution, and you seem to be doing something else. See Wikipedia $\endgroup$ – Olli Niemitalo Nov 16 '18 at 7:21
  • $\begingroup$ @OlliNiemitalo thanks much for the pointer, I just edited the post accordingly! $\endgroup$ – Darius Petermann Nov 16 '18 at 7:26
  • $\begingroup$ Related question: Choosing the right overlap for a window function $\endgroup$ – Olli Niemitalo Nov 19 '18 at 13:23
2
$\begingroup$

Let's assume continuous time (rather than discrete time).

If you do not process the windowed data at all, you would like the output (the sum of the windowed frames) to be equal to the original signal. Allowing scaling of the output by a constant scaling term, this is only possible if the sum of all of the time-shifted window functions is constant over time.

A typical window function (including Hann) is a sum of harmonic cosine terms. Let's assume you only use such windows. At the end of the window, the cosine terms have derivative discontinuities that can only be "cured" by stitching to the end of the window the start of another window which has identical values of all derivatives. This, and that you probably want the windows to be distributed evenly, requires that the hopsize is $\frac{1}{N}$ window lengths, with $N$ integer. The truncated cosine terms in each of the first $N$ windows are extended to complete cosines by concatenation of every $N\text{th}$ window.

The sum $f(T)$ of the windows distributed this way is:

$$f(T) = \sum_{n = 0}^{N - 1}\sum_{h = 0}^{H - 1} a_h\cos\left(2\pi h\left(T-\frac{n}{N}\right)\right),$$

where $h$ is the harmonic number, $H$ is the number of harmonics in the window function, $a_h$ are the window function specific coefficients, $T$ is time expressed so that a time difference of $1$ means one window length. $h$ and $n$ are the running harmonic and hop numbers.

Let's try window densities $N = 1\dots3$ with a window that has two harmonics ($H=2$). First $N=1:$

$$f(T) = a_0 + a_1\cos(2\pi T)$$

then $N=2:$

$$f(T) = a_0 + a_1\cos(2\pi T) + a_0 + a_1\cos\left(2\pi \left(T - \frac{1}{2}\right)\right) = 2a_0$$

and $N=3:$

$$f(T) = a_0 + a_1\cos(2\pi T) + a_0 + a_1\cos\left(2\pi \left(T - \frac{1}{3}\right)\right) + a_0 + a_1\cos\left(2\pi \left(T - \frac{2}{3}\right)\right) = 3a_0$$

This could be for example the Hann window with $a_0 = \frac{1}{2}$ and $a_1 = \frac{1}{2}:$

enter image description here
Figure 1. Hann windows sum to constant with a hopsize of 1/2 or 1/3 window lengths, but not with a hopsize of 1 window length.

Although it is not as evident as with the 1/2 hopsize, the $1\text{st}$ harmonic cosines cancel each other completely with the 1/3 hopsize. The cancellation can be illustrated by representing the phases and amplitudes of the harmonic terms as complex numbers:

enter image description here
Figure 2. Complex number representation of the amplitudes and phases of the 1st harmonic terms in shifted concatenated Hann windows, with a hopsize of $1/3$ window lengths. The rotational symmetry of the pattern means that the first harmonics cancel, because the sum of the three complex numbers must be zero.

For window functions with $H$ harmonic cosine terms, if we tabulate for different window densities $N$ the factors appearing in front of each $h\text{th}$ harmonic cosine term in $f(T),$ a pattern emerges:

$$\begin{array}{r|lllllllllll}&\rlap{h}\\ N&0&1&2&3&4&5&6&7&8&9\\ \hline 1&1&1&1&1&1&1&1&1&1&1\\ 2&2&0&2&0&2&0&2&0&2&0\\ 3&3&0&0&3&0&0&3&0&0&3\\ 4&4&0&0&0&4&0&0&0&4&0\\ 5&5&0&0&0&0&5&0&0&0&0\\ 6&6&0&0&0&0&0&6&0&0&0\\ 7&7&0&0&0&0&0&0&7&0&0\\ 8&8&0&0&0&0&0&0&0&8&0\\ 9&9&0&0&0&0&0&0&0&0&9\end{array}$$

The factors in the 2-term window example appear in the two first columns and three first rows.

For $H \le N,$ all harmonics except for the $0\text{th}$ harmonic cancel completely. This means for example that the 4-cosine-term ($H = 4$) Blackman–Harris window could be used with a 1/4 hopsize or greater ($N \ge 4$), requiring only an output scaling by $\frac{1}{4a_0} = \frac{1}{4\times0.35876}.$

$\endgroup$
  • 1
    $\begingroup$ Thank you for the extensive and clear answer, Ollie. In my mind, an overlap greater than 50% inevitably meant that amplitude oscillation would occur, I was thus overthinking the solution needed to fix the problem. i/2N totally did the trick. $\endgroup$ – Darius Petermann Nov 16 '18 at 15:57
  • $\begingroup$ @DariusPetermann I rewrote the whole answer. The situation is better than I thought! You have more choices with the overlap, and with enough overlap, any cosine-sum window (not just Hann) will work perfectly. $\endgroup$ – Olli Niemitalo Nov 19 '18 at 11:04
  • $\begingroup$ Hi Olli, thank you so much for adding such great details to your answer. This is far better and clearer than most answers I was able to find online, especially since it discusses other window types else than Hann (I am using Hamming, Blackman and 3-term-Blackman-Harris as well). As a quick follow-up question, would you be able to point me to any resources that I could use as a starting point next time I am running into a similar issue ? - Thanks again! $\endgroup$ – Darius Petermann Nov 20 '18 at 22:08
  • $\begingroup$ @DariusPetermann Not sure what similar issue, but for some literature references on lapped processing, see answers to Choosing the right overlap for a window function and FFT with asymmetric windowing?. $\endgroup$ – Olli Niemitalo Nov 21 '18 at 10:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.