Audio Denoising - Method which Extends the Naive Noise Reduction

Given an audio signal x[n] sampled at 44.1khz (let's say 1 minute of music or speech) and a noise template noise[n] (let's say 2 seconds, for example tape hiss), this might be the simplest STFT-based noise reduction algorithm :

noisetemplate = np.abs(stft(noise))).mean(axis=0)

xSTFT = stft(x)
outSTFT = np.zeros_like(x)

for t in range(xSTFT.shape):              # process each STFT frame
a = np.abs(xSTFT[t, :]) - noisetemplate  # spectral substraction for each frequency bin
a = a * (a > 0)                          # if negative value, make it 0
outSTFT[t, :] = xSTFT / np.abs(xSTFT) * a

# inverse STFT with overlap-add, etc.


It works ok, but I think we can do better.

What is a step further / a little bit better STFT-based noise reduction than this naive spectral substraction ?

Note: I've read a few things about Wiener, but I'm still unable to modify the previous code to turn it into Wiener filtering...

• Designing a Wiener filter requires that one knows the complete statistical behaviour of the noise source. When this complete knowledge is not possible, an estimation will be used instead. The statistical characterisation of the noise source can be estimated (i.e. computed) from a block of noise-alone samples. – Fat32 Jul 13 '16 at 9:16
• @Fat32 I only have one second of noise source, but this is nearly 40 FFT frames (can vary depending the FFTsize and overlap)... So this would be enough right? – Basj Jul 13 '16 at 9:35
• The answer could only be provided had we known the true characteristics of teh noise and your method of estimating it. But the practical answer is just try, to see if it is enough... – Fat32 Jul 13 '16 at 9:54
• the true characteristics is the process which generated the physical noise source, that we want to knoe but cannot completeley determine based on th evailable data, hence we can only estimate it based on those samples. sorry but at the moment I cannot provide a pseudo code solution that could fit to your problem framework... – Fat32 Jul 13 '16 at 11:13
• Hi, I am not sure I understood the model. You have samples of Audio which contains a signal and Hiss Noise. On the other hand you have clean Hiss samples which based on you want to clean the audio? Could you put samples? Thank You. – Royi Jul 16 '16 at 8:35

You could try different approaches all together.

For instance in Image Processing there is an efficient method for Denoising called Non Local Means.
It has an extension to Audio - Non Local Means for Audio Denoising by Arthur Szlam.

Looks like a form of scalar thresholding in the STFT domain to me, with a diagonal operator.

But (from Audio Denoising by Time-Frequency Block Thresholding, 2008):

Removing noise from audio signals requires a non-diagonal processing of time-frequency coefficients to avoid producing "musical noise." State of the art algorithms perform a parameterized filtering of spectrogram coefficients with empirically fixed parameters. A block thresholding estimation procedure is introduced, which adjusts all parameters adaptively to signal property by minimizing a Stein estimation of the risk. Numerical experiments demonstrate the performance and robustness of this procedure through objective and subjective evaluations.

If you dare to look at more recent stuff, try Consistent Wiener Filtering: Generalized Time-Frequency Masking Respecting Spectrogram Consistency, 2010

[EDIT] The algorithm is a form of scalar thresholding. For each coefficient, you either keep a coefficient whose magnitude is above the threshold, or cancel it. It is generally better to perform block thresholding: you take a neighborhood of one coefficient, compute the energy (or some weighted norm) of all the coefficients, and modify (shrinkage) this center coefficient with respect to a threshold. I have not used it for audio, but you can find a generic formula for that in Equation (17) of A Nonlinear Stein Based Estimator for Multichannel Image Processing.

• Can you explain a little bit how the algorithm works? What is the idea behind it? What is diagonal / nondiagonal (in this context)? (of course I know diagonal matrices, operators, etc. in math, what is it in this context) ? – Basj Feb 9 '17 at 20:47
• In $D\times F$, $D$ diagonal, you just multiply $F$ in a component wise way. For instance, convolution in time becomes a diagonal product in frequency. – Laurent Duval Feb 11 '17 at 13:52