STFT frequency domain processing destroys window shape, makes synthesis glitchy

Question

I have a signal processing chain which goes:

• 1. Every 256 samples, take 512 samples of input buffer
• 2. Apply a Hann window to the samples
• 3. Copy the windowed samples into the real components of a complex array
• 4. FFT the complex array, obtaining complex spectral analysis data
• 5. Modify the spectral analysis data
• 6. IFFT the modified spectral analysis into a complex array
• 7. Overlap-write the real components of the complex array into an output buffer at 256-sample intervals.

If stage 5 does nothing, the input signal is perfectly reconstructed. The data being overlap-written into the output buffer at (7) has the Hann window shape applied at (2), so the overlaps merge smoothly.

If in (5) I try to apply a spectral delay, by adding in frequency components captured in previous windows, the Hann window shape is lost - the output data no longer tapers to 0 at both ends - and the overlaps do not merge smoothly. Horrible crackles ensue.

I can reapply a Hann window to the output data before overlap-writing, but then I get a sort of "warbling" effect due to the fact that I've now effectively windowed the data twice.

Ideally the stage 5 processing would leave the window shape intact - but it's unclear how to do this when changing the spectral analysis data. (Note that I'm already ensuring that conjugate symmetry is preserved). Is there a missing step? Do I need to apply some phase adjustment to the delayed frequency components?

Answer 1

Frequency domain processing is complicated primarily because of "time domain aliasing".

Most frequency domain manipulations can easily be expressed as a linear filtering operation, i.e.

$$H[k] = \frac{Y[k]}{X[k]}$$

where $X[k]$ is the input spectrum, $Y[k]$ the resulting spectrum and $H[k]$ the transfer function of the "effective" filter. The impulse response of that filter is given by the inverse DFT of the transfer function, i.e.

$$h[n] = \text{DFT}^{-1}\left\{ H[k] \right\}$$

Multiplication in the frequency domain is equivalent circular (not linear) convolution in the time domain. If $h[k]$ has non trivial time extension, than the linear convolution of signal and impulse response will longer than the DFT size and the overage will warp around to the beginning of the buffer. That creates the crackles and that's time domain aliasing.

If you do not change the spectrum, the effective transfer function becomes $H[k] = 1$ and the impulse response is $h[n] = \delta[n]$. That has a length of 1, so no aliasing (and crackle) occur.

The standard solution for linear time invariant filter is zero-padding using Overlap-Add or Overlap-Save algorithm.

For time variant filters, things become much more complicated. In essence you need to control your spectral manipulation so that the equivalent impulse response stays causal and temporally compact. As far as I know, there is no "one size fits all" solution for this. There are a variety of methods (increase zero pad, decrease hop size, symmetric square root windows for both transforms, controlling the speed of "time variance" in the spectra domain, etc.) but the best choice depends a lot on the properties of your signals, the nature of your spectral manipulation and the requirements of your application.