# Diagonal vs non diagonal processing for audio denoising

I am going through a technique for audio denoising by block thresholding. Here is the paper:

http://www.di.ens.fr/~mallat/papiers/AudioBlock.pdf

According to the definition in the paper for diagonal estimators, Simple time-frequency denoising algorithms compute each attenuation factor only from the corresponding noisy coefficient and are thus called diagonal estimators.

and for non diagonal estimators: To reduce musical noise as well as the estimation risk, several authors have proposed to estimate the a priori SNR with a time-frequency regularization of the a posteriori SNR . The resulting attenuation factors thus depend upon the data values for in a whole neighborhood of (l,k).

I am not quite able to comprehend these definitions, and the difference. The paper also claims that non-diagonal processing is better than diagonal processing techniques. Could someone explain the difference in a layman's terms?

I find the diagonal term a bit confusing. Sometimes, for wavelets, people separate scalar and vector or block thresholding. [EDIT] Now I believe I understand it. If you put all your samples or coefficients in a long 1D vector, then multiplying it by a diagonal matrix yield a scalar operation. With non-diagonal, you allow cross-terms.

For structured signals, or images, or transformed signals (in a wavelet or a time-frequency domain), neighboring samples or coefficients often possess some local correlations, that you want to exploit in denoising. So assume you have an observation $r(k)$ at location $k$ (could be multidimensional), and look for an estimator $\hat{s}(k)$ of some underlying signal $s(k)$, under a noisy model $r(k) = s(k)+n(k)$. In the original domain, you could for instance apply a scalar mask $f(k)$ on the observation: $\hat{s}(k) = f(k).r(k)$, especially if you know that at some location you have signal, at other noise. Windowing is a typical example. More generally, you can apply a function $f$: $\hat{s}(k) = f(r(k))$. If you assume local correlations, you can borrow information for close samples. For instance, average filtering can be $\hat{s}(k) = \frac{1}{3}(r(k-1)+r(k)+r(k+1))$.

In other words, $f$ now apply over a domain, gathered in a vector $v(k)= [r(k-1),r(k),r(k+1)]$. It can be called Reference Observation Vector (ROV). Then $\hat{s}(k) = f(v(k))$. So average filters are sort of non-diagonal processing.

Traditionnaly with wavelets or time-frequency, the transform compacts or sparsifies noisy data, and data features tends to become relatively higher than the noise, compared to the original domain. So the previous idea of a scalar mask works better. You can choose $f$ to keep the value of $r(k)$ if $|r(k)|$ is above a threshold $t$, and set it to zero if below. This $f$ is discontinuous. let us write it $f(x) = \mathrm{sign}(x) (|x|-t)_+$ This is called scalar wavelet shrinkage or thresholding, scalar because you apply it on a single wavelet coefficient, illustrated in the center plot.

This idea was already used in the Fourier domain for noise reduction in speech. You can have smoother transistion, if you attenuate a bit the higher values, which give you soft thresholding, and you have many more shrinkage functions (garrote, firm) for scalar thresholding.

The next idea is to combine all of the above. You can modify one coefficient with respect to a certain norm of a surrounding $\|v(k)\|$, as:

$\hat{s}(k) = f(\|v(k)\|)r(k).$.

In wavelets, the ROV $v(k)$ can be more complicated. It can gather neighbors of the coefficients, but also coefficients from other scales.

The rationale is that, if a coefficient is small, it might be useful to preserve it anyway. If its ROV has high norm (think of energy), the coefficient might small by mistake, and is not noise.

The nice think is that the Stein Unbiased Risk Estimator theory allows you to estimate something on the signal $s(k)$ that you do not know by using only the observation $r(k)$, using some technical conditions, which allows you to find, automatically, the best threshold $t$, or shape paramaters of the function $f$. Such methods yielded almost the state-of-the-art on wavelet denoising. I am not familiar with audio. However, for multispectral images, here is an example:

• a satellite image of the port of Tunis:

• its noisy version, with heavy Gaussian noise:

• the denoised version:

While not perfect, it was able to recover details, almost invisible from the noisy image, with very few parameters to set (only the shape of the ROV). This answer is a short version of what you can read in A Nonlinear Stein-Based Estimator for Multichannel Image Denoising, 2008.

Why can block denoising be better that scalar? Because scalar is a specific case of block (non-diagonal) processing, if you take the ROV to be restricted to the coefficient $r(k)$ itself. If you choose the time-frequency or time-scale method well, and the other parameters (block shape and size), the block processing is generally (much) better.

• Hey! I didn't know you were an IEEE member! Well done! :-) – Peter K. Apr 20 '16 at 16:05
• @ Peter K. Yes, it means I am entitled to author papers like this one :) cs.tut.fi/~bregovic/files/ieee.pdf – Laurent Duval Apr 20 '16 at 17:07
• Don't laugh! A colleague of mine had a proof rejected by the IEEE Trans on SP with "but that all just follows from the equality axiom." Doesn't everything?!?!? – Peter K. Apr 20 '16 at 17:14
• @Peter K. Now I see I have had two papers of you in my reference database (with Lovell, 1990, 1991) for a while. I do apologize for the small number, and will cure that right away. – Laurent Duval Apr 20 '16 at 17:15
• @Peter K. a whole world of science dwells in the "just", I guess – Laurent Duval Apr 20 '16 at 17:17

just a cursory look at the paper seems to confirm it's a wavelet transform thingie:

A time-frequency transform decomposes the audio signal over a family of time-frequency atoms $\{ g_{l,k} \}_{l,k}$ where $l$ and $k$ are the time and frequency (or scale) localization indices.

so i think your threshold is about throwing out (or attenuating) particular atoms that have energy below some noise-level threshold. anyway that thresholding thing can be sorta gradual, you're not multiplying by a step function of the energy of each of the atoms. it can be smoother than that.

with the time-frequency array (or map) of the atoms, i dunno what significance any diagonal is, with respect to denoising. each diagonal will sorta correspond to a linearly-swept frequency "chirp" signal. i dunno why one would be discriminating w.r.t. them.

you can make your threshold for gating somewhat proportional to the level, localized both in time and frequency (to different degrees, if you want), in that time-frequency map. i.e. blur or average the levels close to whatever point in both time and frequency. use that low-pass-filtered level to determine the threshold, and (smoothly) gate the atoms off that fall below that locally determined threshold.

i dunno.