4
$\begingroup$

I have sucessfully implemented a STFT (+inverseSTFT with perfect reconstruction of original signal, with overlap add, etc.) in order to work on audio files.

Using this STFT / iSTFT framework, I have tested :

  • Basic "noise reduction" algorithm that works by spectral subtraction of a noise template -> it works very well !

  • Hi-pass filtering by zeroing the lowest bins (or, better idea : multiplying these lowest bins by a smooth window, like Hann). It works (the filtering is good to my ears), but the spectrogram is not what I want :

enter image description here

I would prefer (for the low frequency 0hz-300hz) something like :

enter image description here

What should I do in the STFT array in order to do a good hi-pass filtering? (i.e. having a clean spectrogram in the low frequencies like the second picture here)

PS : here is the STFT scheme I use in Python : http://pastebin.com/MdycVLQk

$\endgroup$
  • $\begingroup$ Is there a particular reason you want to do filtering of noise in the Time-Frequency domain, (STFT/iSTFT framework), and not just use a simple filter? If you can work with the original audio file, you can use efficient filtering techniques. If you are forced to remove more complicated noises, then you can work in the TF-domain with STFT/iSTFT non-linear methods. Which one suits you best? $\endgroup$ – Tarin Ziyaee Dec 3 '13 at 20:55
  • $\begingroup$ I use STFT/iSTFT in order to do Noise reduction (I use a "noise template", and then I can do spectral subtraction in the STFT framework). This is a very classical topic, and at the end, OLA is used to reconstruct the signal. It works. BUT I also want to hi-pass the signal, and so I wondered if it was possible to do this hi-pass filtering in the STFT framework (if possible, I'd want to use the STFT framework as much as possible in order to avoid lots of totally different processings!) $\endgroup$ – Basj Dec 3 '13 at 21:03
  • $\begingroup$ Ok I see. In your 'noise template', you mean that you are given what the shape of your noise will look like in the (modulus) of Time-Frequency domain? $\endgroup$ – Tarin Ziyaee Dec 3 '13 at 21:08
  • $\begingroup$ Yes. I use a part of the sound where there is "Noise only" and then I compute its STFT. I do averaging along frames, and I get an average "modulus" for each bin : this is the noise floor. Then I do some sort of subtraction of this noise floor to the signal I want to denoise. Read for example Ephraim Malah algorithms, etc. $\endgroup$ – Basj Dec 3 '13 at 21:12
  • $\begingroup$ Ok, then in this case, and for more complicated noises in the Time-Frequency domain, yes you can use STFT/iSTFT with OLA reconstruction, or LSE reconstruction. OLA reconstruction of an STFT is actually a heuristic, but can still be used. The LSE has a firmer theoretical basis. Both can work however. Also note that the OLA in time-domain convolution is NOT the same as what is meant in your OLA of the iSTFT. They are very different. All in all, sounds good. I wanted to understand why you were doing it in the STFT/iSTFT domain and not time domain, and your reason is very good. $\endgroup$ – Tarin Ziyaee Dec 3 '13 at 21:18
2
$\begingroup$

The techniques you are referring to advanced techniques, and are generally known as "Denoising via STFT-masking".

First, you create an STFT matrix, and mask its modulus (that is, the absolute magnitude of the STFT matrix) by whatever weights you chose. You can pick binary masks, (null out Time-Frequency bins you do not want while retaining Time-Frequency bins you do want), or you can perform soft-masking, in which the weights are not binary, but follow some metric commensurate with your data in the Time-Frequency plane.

Next, you apply an inverse STFT, and this can be done by finding the Least Squares Estimate of a time-domain signal, that would give the modified (complex) STFT after you have performed the masking. (See this paper here and Equation 6 of Griffin and Lin for a Least-Squares treatment of inverse STFT transforms via Time-Frequency bin nulling). (In your STFT inversion, you must also make sure that the sum of the window co-efficients across summed frames add up to unity, so often times half cycle sin window is used with 50% overlap in the STFT analysis. This is not the only solution however).

Therefore in its simplest incarnation, filtering out high-frequency components in the STFT domain means simply setting those STFT bins to zero, and then applying the Least-Squares Optimization algorithm to re-synthesizing your time-domain vector. (Similarly, if you want to do a high pass, you can null out, or softly change, the STFT coefficients in the low bands of the matrix). Note that the LSE will be intrinsic to the ISTFT if you use the Griffin paper.

$\endgroup$
  • $\begingroup$ Yes it's exactly what I use : STFT, then modify some values (you're right : only the modulus of some values) in the matrix, and then inverse STFT (with a simple Overlap-Add method : I use a window such that the shifted windows add up to unity). I don't understand why is LSE needed for inverse STFT. The inverse can be done without LSE but rather with a simple overlap-add, so why is it interesting to use LSE ? $\endgroup$ – Basj Dec 3 '13 at 19:47
  • $\begingroup$ @Basj The LSE - (or any other norm for that matter) optimization is required, because an arbitrary STFT is not necessarily a valid STFT, in the sense that there is no signal whose STFT is given by what is presented. $\endgroup$ – Tarin Ziyaee Dec 3 '13 at 19:58
  • $\begingroup$ I don't understand : if I have perfect reconstruction with my inverse STFT, then this means that my STFT is valid, don't you think so ? Here it is : pastebin.com/MdycVLQk $\endgroup$ – Basj Dec 3 '13 at 19:59
  • $\begingroup$ @Basj Yes, if you do not modify your STFT, then it is a valid STFT, and maps to a given signal. Only if you modify your STFT, then there is no guarantee that it remains valid, and LSE or other criteria are then used to estimate what signal might have given rise to your signal. $\endgroup$ – Tarin Ziyaee Dec 3 '13 at 20:00
  • $\begingroup$ Ok I see... What is the definition of valid then ? Do you think this one pastebin.com/MdycVLQk will produce valid STFT ? $\endgroup$ – Basj Dec 3 '13 at 20:03
2
$\begingroup$

You need to analyze the impulse response of your frequency domain modifications (treat it as a filter), and make sure the length of your overlap (using overlap save/add) is at least as long as the significant portion of this impulse response (above your desired noise level). This may require a longer FFT, or a frequency domain modification with wider "softer" transition-bands (or both).

$\endgroup$
  • $\begingroup$ Thanks for answer. As already pointed by Jazzmaniac, it seems that I should zero pad.... But where should I zero pad? Do you could show me the line where I should add zero padding here : pastebin.com/MdycVLQk ? More generally, it seems that I would need to read a good reference about : how to avoid circular convolution and have linear convolution (is this the point here?). Do you have a book reference for this ? $\endgroup$ – Basj Dec 3 '13 at 21:57
  • $\begingroup$ Instead of zero-padding, try a much longer FFT (and IFFT for your fast convolution result) with much more overlap in your overlap add/save filter. $\endgroup$ – hotpaw2 Dec 3 '13 at 23:01
  • $\begingroup$ Where a longer FFT (pastebin.com/MdycVLQk) ? I'm still a bit lost ;) $\endgroup$ – Basj Dec 3 '13 at 23:10
  • $\begingroup$ I've never understood why people still bother with this stuff. It's close to useless. $\endgroup$ – Peter K. Dec 4 '13 at 4:35
  • $\begingroup$ Hi @PeterK, which stuff are you talking about : using a zero-padded STFT / iSTFT ? Or even using STFT filtering ? I recall that I want to do STFT hipass-filtering just because I already have a STFT/iSTFT framework in my processing chain (for Noise reduction), so I would like to re-use it while filtering... $\endgroup$ – Basj Dec 4 '13 at 7:03
0
$\begingroup$

You cannot just multiply the STFT frames with your filter response and invert the fourier transform to get a filtered signal. Any multiplication in frequency domain is a convolution in time domain. And the frequency response you use has a certain time domain kernel associated with it. This kernel will wrap around at the two frame boundaries when you convolve like that, because FFT convolution is inherently circular. Specifically steep filters like your high pass will have a rather long range effect on samples and introduce the artifacts you're seeing.

The solution is to use overlap add or overlap save convolution schemes that turn the circular convolution into linear convolution.

$\endgroup$
  • 1
    $\begingroup$ You can in fact multiply STFT bins with whatever co-efficients you like and then inverse transform, provided that the resulting gibbs phenomena is either acceptable or negligible. LSE techniques exist to mitigate its effects. This is an advanced field known as Time-Frequency Masking. See my answer for some links. $\endgroup$ – Tarin Ziyaee Dec 3 '13 at 19:26
  • $\begingroup$ What do you mean by using an overlap add scheme that turn circ conv into linear conv? Here is the STFT scheme I use : pastebin.com/MdycVLQk (simplified, note that the 3 or 4 first frames won't be perfectly reconstructed, but this can easily be solved with a longer code) $\endgroup$ – Basj Dec 3 '13 at 19:56
  • $\begingroup$ @user4619, I disagree. What you propose is not suitable for the task at hands. The OP obviously wants to realize a time frequency filter and that implies a convolution. Of course you can first make an error and then try to cover it up by trying to minimize it afterwards, but that really misses the points and makes thing much more complicated than necessary. If you want real time frequency filtering you wouldn't need that either. The theory of Weyl symbols already does that without the need to use any kind of fitting or other approximate solutions. $\endgroup$ – Jazzmaniac Dec 3 '13 at 20:19
  • $\begingroup$ @Jazzmaniac This is question in the STFT/iSTFT framework. It is not a question in the time-domain only, (where you can use convolution as you suggested), or wavelet domain, (where you would not need LSE). Thus the answer is also in the STFT/iSTFT framework. Time-Frequency masking is a valid technique, but no one is saying it is the only technique. $\endgroup$ – Tarin Ziyaee Dec 3 '13 at 20:26
  • 1
    $\begingroup$ Zeroing bins has as infinitely long (or very long above your noise floor) impulse response, which will require an infinitely (or very) long FFT so as not to wrap circularly and cause ringing-like artifacts. $\endgroup$ – hotpaw2 Dec 3 '13 at 22:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.