1
$\begingroup$

I am doing STFT (short-time Fourier transform) on real-time audio to filter frequencies by magnitudes (to reduce noise) which have certain drawbacks like noise spikes in reconstructed audio in the time domain. I found that I have to zero pad and try square root hann window to minimize the effects of frequency filtering. I might have to soft mask rather than zeroing frequency.

Currently, I am using frame size as 1024 samples and 50% overlap. I applied the window function and then reconstruct and the audio is ok if don't add the filter. while doing FFT I input 1024 samples array and got 1024 complex array after FFT. then after filtering and inverse FFT, also got a complex 1024 array whose real parts are reconstructed samples. I was simply adding overlapping parts. now to zero pad how many zero's do I need to add and how to handle the overlapping part? if after zero padding my array size becomes 2048(first & last 512 samples are zero) then after doing filtering in freq domain and inverse FFT what does the 2048 arrays will actually mean? where will be the overlapping part and samples?

Improve this question
$\endgroup$
5
  • $\begingroup$ If noise reduction is the only thing you want, you should use time-domain filter instead of STFT. See dsp.stackexchange.com/questions/6220/… $\endgroup$
    – ZR Han
    Feb 8, 2022 at 2:54
  • $\begingroup$ @ZRHan yeah, I am aware of that post. Actually not only noise but also some sidetalks I want to remove. I already designed an adaptive filter which filters by rms. But in a noisy or office environment this can't do much as sidetalk's rms is also high. But through frequency power filter I can achieve this. Thats why I came this long way from rms filter to frequency filtering. Now I just want the unnecessay noise spikes to be removed that are being reconstructed after frequency filtering. $\endgroup$
    – Nafiul Alam Fuji
    Feb 8, 2022 at 3:51
  • $\begingroup$ Ok I see. But why did you get complex IFFT results? If the frame size is 1024, the FFT size should be 2048 for overlap add method. Maybe you can share the pseudo code for better understanding. $\endgroup$
    – ZR Han
    Feb 8, 2022 at 4:07
  • $\begingroup$ actually I am new to this FFT tings. if I simply fourier a array samples of 1024 size I should get 1024 sized complex arrays? If I use overlapping windows like 0-1023, 512-1536 like 1024 sized arrays for fft I should still get 1024 sized complex array? then after inverse fft I should use overlap-add or save? here I am confused. Though if I simply add those overlapping windowed samples (512-1024), the reconstructed audio looks fine. but problem occurs after adding filters $\endgroup$
    – Nafiul Alam Fuji
    Feb 8, 2022 at 4:14
  • $\begingroup$ You should have a look at this article A weighted overlap-add method of short-time Fourier analysis/Synthesis and this page ccrma.stanford.edu/~jos/sasp/WOLA_Processing_Steps.html $\endgroup$
    – ZR Han
    Feb 8, 2022 at 5:26

1 Answer 1

Reset to default
2
$\begingroup$

What you are trying to do is time-variant frequency domain processing. Unfortunately this is quite tricky and mathematically complicated.

The main problem you will have to solve is time-domain aliasing. The modification of the spectrum in each frame is basically a filter, which has a corresponding impulse response in time. If you do nothing to constrain the length of this filter you get the following situation:

Let's assume you have a frame length of $N$ and with zero padding an FFT size of $2N$. You create a spectral mask to implement your filtering. The inverse FFT of this mask is the impulse response of the filter. This will in general have a length of $2N$ samples since the mask has also $2N$ samples. The length of the convolution of signal and filter than will have a length of $3N-1$ samples. However your FFT frame is only $2N$ samples long. Since multiplication in the frequency domain implements circular convolution, your extra $N-1$ samples will wrap around and be added to the first half of your output frame. That's time domain aliasing and it sounds really, really bad.

There are multiple ways to deal with this, but there is no "one size fits all" solution and typically requires a lot of tweaking to get to the optimum set of trade offs for your specific requirements.

Things that can be tweaked:

  1. Frame size
  2. FFT length and/or amount of zero padding. Longer FFTs give the filter impulse response a better chance to decay a the edges.
  3. Hop size
  4. Double windowing: apply both an analysis and a reconstruction window
  5. Choice of window: makes sure that the product of both windows overlapped and summed as given by your frame and hop size actually adds to 1. Preferably to EXACTLY one (which a 50% overlapped Hanning window doesn't).
  6. Frequency domain smoothing. The narrower the frequency domain features are, the longer the impulse response will be (in the sense of more energy far away from the center). There are multiple ways of doing this. Either apply a sliding window in the frequency domain or simply do an inverse FFT, window the impulse response making sure that N samples are zero and do an FFT again. Of course this will somewhat change the frequency response of your filter. It's a trade-off.
  7. Consider making the filter minimum phase instead of zero phase. That makes it causal and also tends to reduce the amount of aliasing since the impulse response has more time to decay.
  8. Slow down how fast any frequency bin can move over time. Apply a first order lowpass to the "gain vs. time" function.
Improve this answer
$\endgroup$
14
  • $\begingroup$ Thanks for the info. I have 2 basic ques in mind though I am not sure it sounds silly or not. let's say I have a frame size of 1024 and decide 50% overlapping. so should this 50% overlapping be actually the zero-padded ones only? like I can have real sample frames of (0-1023), (1024 - 2048) and then make these frames zero-padded the same amount to make frame size 2048 and do 50% overlap (to later do overlap-add). or should I consider real sampled frames as (0-1024), (512-1536), and after zero padding and making the frame size as 2048 is called 50% overlap? is the 1st one or 2nd one correct? $\endgroup$
    – Nafiul Alam Fuji
    Feb 9, 2022 at 2:19
  • $\begingroup$ 2nd is about window function. if my real frame size is 1024 and apply window function to it, it's still 1024 size right? then after apply 1024 zero's to the end and make it 2048 frame size with zero padding? then in the spectral domain with 2048 sized complex array lets say I am simply scaling them 0.1 if their magnitude is below a certain threshold. how can this increase the size greater than 2048 as I am simply updating their values rather than multiplying them in matrix? I saw size increases when I multiply them with function matrixes when calculating convolution. but is this the same? $\endgroup$
    – Nafiul Alam Fuji
    Feb 9, 2022 at 2:26
  • 2
    $\begingroup$ Q1: One way to start would be with a hop size of 512, a window length of 1024, and an FFT size of (at least) 2048. So advance read pointer by 512, grab1024 samples, apply window, zero pad to 2048 , apply FFT. Make sure that you get input = output if you don't change anything in the frequency domain $\endgroup$
    – Hilmar
    Feb 9, 2022 at 14:23
  • 1
    $\begingroup$ Q2: Scaling them by 0.1 is the same as multiply with a filter that has a magnitude of 0.1 at this frequency and 1 everywhere else. So manipulating individual frequencies creates very sharp transients in in the frequency domain which creates a very long and ringy impulse response. For example your gain at 1kHz is 0.1 but it's 1 at 999Hz and 1001 Hz. That's a VERY sharp filter. $\endgroup$
    – Hilmar
    Feb 9, 2022 at 14:26
  • $\begingroup$ I just tried 512 frame size with hop size of 192. so unique 320 samples and 192 overlapping samples per frame. only first frame has 192 zero padding in front. then applied window function,applied STFT,applied inverse STFT, applied inverse window function and reconstruct using overlap save. but reconstructed audio is little distorted. but if I turn off the windowing and inverse windowing I am getting exact audio. why can that happen? is my track ok? I tried applying hann and square root hamming window and inverse later while reconstructing. $\endgroup$
    – Nafiul Alam Fuji
    Feb 9, 2022 at 14:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.