4
$\begingroup$

I have the original signal, a .wav file and by blind source separation, i have reconstructed the signal. How can I calculate the SNR in this case? or is it good to use any other performance parameters? What should be an acceptable value of SNR?

$\endgroup$
5
$\begingroup$

The common practice in the source separation community is to report 3 metrics defined in Vincent et al's paper, called SDR (signal to distortion ratio), SIR (signal to interferences ratio), and SAR (signal to artifacts ratio).

The reason for this is that in source separation applications, there can be two types of "noise" in the reconstructed signal: noise due to mis-separation, called interferences (for example, if you want to extract the vocals from a song, this would be a residual of the background music), and noise due to the reconstruction algorithm itself, called artifacts (for example, glitches due to the STFT phase estimation process). The former is sometimes referred to as "crosstalk" in other contexts.

There is usually an application-dependent trade-off between the two - for example in a Karaoke application (vocal removal), it is not a big problem if the audio accompaniment has still a few traces of the original voice (low SIR), as long as the result is pleasing to the ears and glitch-free (high SAR). Most separation algorithms have "knobs" that will allow you to balance SAR and SIR. For example, increasing the STFT window size tends to increase SIR but decreases SAR.

The estimation of these metrics is described in the paper by Vincent et al, and implemented in the bss_eval toolbox. Of course, access to the original individual sources you want to recover is necessary! You will have to evaluate your algorithm on a data-set for which the "right answer" (original individual track) is known.

Regarding the acceptable value of the metrics, you need to compare your results with those published in the academia for the same problem you are attempting to solve. Keep in mind that these are objective metrics, and as such, do not take into account the perceptual quality of the noise. It might be possible that an algorithm with a -12dB SAR will be more pleasing to the ears than an algorithm with a -24dB SAR if the former produces artifacts well spread in time and below the masking threshold of the reconstructed signal. SDR/SIR/SAR can be used in development to rule out bad ideas and tune your algorithms, but for a commercial application, listening tests are ultimately necessary.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Some explanations, tasks and results on Audio Source Separation are gathered on the Signal Separation Evaluation Campaign (SiSEC). Note also that an interesting measure is the improvement in SDR: compare to the SDR obtained with the mixture used for all the "estimated sources" (the most trivial separation system, actually these SDRs are the original source-to-interference ratios). As @pichenettes said, the presence of artifacts can lead your results to be worse than the trivial system: a positive improvement is therefore already an acceptable result! :D $\endgroup$ – Jean-louis Durrieu Jun 18 '13 at 21:24
  • $\begingroup$ I just trained a model that sounds pretty good except for a mysterious demon voice that I hear on a lot of the test set tracks. By demon voice I mean an extremely low sound that resembles speaking. I'm assuming this is due to artifacts as there was no voice that low in the original data. Would this be correct? $\endgroup$ – Luke Aug 27 '18 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.