How to extract vocal part from stereo audio signal?

Question

I'm now processing MP3 file and encounter this problem. My MP3 is stereo encoded. What I want to do is extract vocal part for further processing(whatever mode of output signals, mono or stereo are both OK).

As far as I know, audio is encoded into different dis-joint sub frequency bands in MP3. I think I can limit the signals to the vocal range through high-pass/low-pass filter with cutting-off frequency proper set. However, result must contain parts of pure music signal in this case. Or after googling, I think I may calculate the background signals first(by inverting one channel adding with signals from the other channel assuming vocal part is centered in the stereo audio called phase cancellation). After this transformation, the signal is mono. Then I should merge the original stereo into mono from which extracting the background signal.

Given the effectiveness, which one is preferred(or any other solutions:)? If the 2nd one, let two channels A and B, will (B-A) or (A-B) used when compute the background? As with merging two channels, does the arithmetic mean accurate enough? Or I can downsample each channel by a factor of two and interleave the downsampled signals as mono result?

Thanks and best regards.

Answer 1

First of all, how the data is encoded in a mp3 file is irrelevant to the question unless you aim at doing compressed-domain processing (which would be quite foolish). So you can assume your algorithm will work with decompressed time-domain data.

The sum / difference is a very, very basic trick for vocal suppression (not extraction). It is based on the assumption that the vocals are mixed at the center of the stereo field, while other instruments are panned laterally. This is rarely true. L-R and R-L will sound the same (the human ear is insensitive to a global phase shift) and will give you a mono mix without the instruments mixed at the center. The problem is, once you have recovered the background, what will you do with it? Try to suppress it from the center (average) signal? This won't work, you will be doing (L + R) / 2 - (L - R), this is not very interesting... You can try any linear combinations of those (averaged and "center removed"), nothing will come out of it!

Regarding filtering approaches: the f0 of the voice rarely exceeds 1000 Hz but its harmonics can go over that. Removing the highest frequency will make consonants (especially sss, chhh) unpleasant. Some male voices go below 100 Hz. You can safely cut whatever is below 50 or 60 Hz (bass, kick), though

Some recent developments in voice separation worth exploring:

• Jean Louis Durrieu's background NMF + harmonic comb > filter model. Python code here.
• Rafii's background extraction approach. Straightforward to code and works well on computer-produced music with very repetitive patterns like Electro, Hip-hop...
• Hsu's approached based on f0 detection, tracking and masking. "A Tandem Algorithm for Singing Pitch Extraction and Voice Separation from Music Accompaniment" (can't find accessible PDF).

Answer 2

Thanks for the reference! You forgot to mention your work on drums enhancement, which may also be of interest for Summer_More_More_Tea's application. Well, that all really depends on what you want to do with it. Do you have a specific "end application" in mind?

I completely agree with pichenettes's above statements. To be complete, I should however say that the vocal enhancement you mentioned has also been used in some works by Matti Ryynänen, on Karaoke track generation, to enhance the results.

To answer your questions:

Given the effectiveness, which one is preferred(or any other solutions:)?

As pichenettes said, neither seems to suit your need: low-pass/high-pass filtering is bound to fail because of the harmonic structure of the human voice (and more generally of any "interesting" sound - i.e. anything beyond sinusoids...).

If the 2nd one, let two channels A and B, will (B-A) or (A-B) used when compute the background? As with merging two channels, does the arithmetic mean accurate enough?

Again, the second method you mention won't do because you can only remove the signal which is in the center, not retrieve it. In other words, even the vocals are in the "center", there is no simple maths to get a vocals only signal.

Or I can downsample each channel by a factor of two and interleave the downsampled signals as mono result?

er... averaging the channels to obtain a mono-channel signal, as suggested above, makes sense, and won't break the spectral characteristics of your signal (assuming the stereo signal is not degenerated). So you obtain a mono signal in which you have, basically, the same musical content as before.

Correctly downsampling each channel means you first apply a low-pass filter (with cut-off frequency of sampling_rate/4 in your case), and then you can safely take every 2 samples. There not much to say however about interleaving the channels thus downsampled: in most general cases, this is breaking the spectral characteristics of your signal. You probably do not want that.

Indeed, the operation of low-pass filtering followed by setting to 0 every 2 samples, and keeping these 0's leads, in the Fourier domain, to "mirroring" the low frequency components that were kept onto the high frequency ones. Remember you signal processing lessons on sampling theory: multiplying by a sequence of impulses (or diracs) results in a convolution with another sequence of diracs in the Fourier domain, i.e., in that case, the frequency spectrum of the signal is repeated (periodized) along the frequency axis, with a period equal to the sampling rate.

Normally, when downsampling, you remove the 0's (because you assume a new sampling rate). But here, keeping them results in very annoying additional high frequency components. Interleaving these signals is not going to correct this.

Well, all in all, the short answer: don't do that. :-)

At last, I might also suggest you to use the GUI I developed for the LVAICA 2012 conference: there is a git repo for it. I am still debugging and improving it, so comments are welcome :D

Hope that helps!