14
$\begingroup$

I have a little karaoke-style app where a user sings 4 lines of a song, with a one second gap between each line. There is no backing music, so it's voice only, hopefully making the problem easier to solve.

I am looking for the most robust way to detect exactly where in my recording the user starts and ends singing line 1, starts and ends singing line 2, etc.

I have cobbled together a simple-minded algorithm that works when there is very little background noise in the recording (like when does that happen?), but it falls to pieces in the presence of the smallest noise.

Can anybody point me towards something more robust?

Improve this question
$\endgroup$
2
  • $\begingroup$ My answer might help you - dsp.stackexchange.com/a/1543/440 $\endgroup$
    – Dipan Mehta
    May 16 '12 at 13:01
  • $\begingroup$ In some recordings the voice is the only instrument that is played with the same amplitude in both channels. You could use this information to extract the voice from a stereo recording. $\endgroup$
    – Jader Dias
    May 29 '12 at 17:13
4
$\begingroup$

If the background noise is white-ish, you could measure spectral flatness and consider it to be voice when the amplitude is above some threshold and the spectral flatness is below some threshold.

Basically you just take an FFT of a chunk of the signal, then divide the geometric mean of the magnitude of the spectrum by the arithmetic mean.

You could also use a band-pass filter to emphasize only the frequency regions where the human voice usually sits (as simple as setting the unwanted regions of the FFT to 0 before measuring the spectral flatness)

Improve this answer
$\endgroup$
4
  • $\begingroup$ Endolith, in your experience, does it make a big difference if you take the magnitude squared VS just the magnitude before you measure the spectral flatness? $\endgroup$
    – Spacey
    May 15 '12 at 18:42
  • $\begingroup$ @Mohammad: I don't have any experience with it. :) I asked a previous question because I didn't understand the spec, and am still not sure which way is correct. I don't think the squaring makes any practical difference, though. If you're just triggering when it passes a threshold, it should react identically whether squared or not (assuming you adjust the threshold to match), so the magnitude-only is computationally cheaper. $\endgroup$
    – endolith
    May 16 '12 at 14:04
  • $\begingroup$ @Endolith, this is a bit of a long shot: do you know how to implement this approach in matlab? I want to test all the approaches mentioned here in matlab (octave actually) to see which is best? $\endgroup$
    – Mike Hogan
    Jun 4 '12 at 4:56
  • $\begingroup$ @MikeHogan: No, I haven't used matlab in a long time. :/ Also that would require doing real work. :) I don't have anything pre-written. Break the signal up into small chunks, do the FFT of each one, and for each FFT, divide the geometric mean of the magnitude by the arithmetic mean of the magnitude. higher numbers are noisy, lower numbers are tonal. $\endgroup$
    – endolith
    Jun 4 '12 at 13:55
3
$\begingroup$

I have used spectral-flux in the past and it seems to work nicely. The basic idea is, create a spectrogram of your signal, across the bands you care about. Let us assume that your frequency is on the y-axis, and your time is on the x-axis, like so.

This means that your spectrogram is a matrix. Each column represents the absolute value of the FFT of one snap-shot in time of your signal, and each row represents how energy from one band changes over time.

Now, simply take the difference of columns. That is, take a column, and subtract from itself the column before it, and do for all columns. (Leaving the start columns alone obviously). Then sum across all the bands. That is, just sum all the rows together.

You will end up with a 1-D signal that codifies your signal onsets. This will tell you where your voice starts.

EDIT:

Now that you have detected onsets, if you want to detect the opposite, (that is, when a signal goes from having activity to none), the spectral flux actually gives you that information. Wherever you have an onset, you will have a positive peak, and wherever you have a 'deset' (for lack of a better word), you will have a negative peak.

I would simply take the first positive peak, and the last negative peak, to mark the total start and stop times of my signal.

Improve this answer
$\endgroup$
13
  • $\begingroup$ Mohammad, what do you mean by "row"? $\endgroup$
    – Mike Hogan
    May 15 '12 at 15:49
  • $\begingroup$ @MikeHogan Please see my edits, I re-wrote the answer. $\endgroup$
    – Spacey
    May 15 '12 at 16:14
  • $\begingroup$ Wouldn't this find onsets of anything, though? Drum hits or other impulsive noise would also be detected. It doesn't differentiate between tonal sounds and noisy sounds. $\endgroup$
    – endolith
    May 16 '12 at 14:17
  • $\begingroup$ @endolith You raise a good point - however I believe that it might still work. My reasoning is that in this scenario, you are in either voice+music state, or just in music state. So when computing the spectral flux, you are really just computing the delta between voice+music and just music. (Of course I would need to analyze it a lot more but thats how I think of it now) :-P $\endgroup$
    – Spacey
    May 16 '12 at 18:28
  • 1
    $\begingroup$ @endolith I just re-read and the OP is saying there is only voice, (apparently its a simple app), so in this case it would just be voice VS nothing. $\endgroup$
    – Spacey
    May 16 '12 at 18:40
2
$\begingroup$

From my experience I would try looking into Mel-Frequency-Cepstrum-Coefficients (MFCCs). MFCCs are fairly easy to implement if you have an FFT available and are used quite commonly in voice processing.

With MFCCs, you should be able to distinguish actual voice data from noise.

Improve this answer
$\endgroup$
1
  • $\begingroup$ @endolith, that link is completely beyond me! Do you know of any open source implementation I can look at, or a more step by step recipe of how it works? $\endgroup$
    – Mike Hogan
    May 29 '12 at 7:42
2
$\begingroup$

"Spectral flux" (also referred to as "spectral difference") is a common method for "onset detection". Basically you take sequential FFTs of the signal and sum the magnitudes of the differences of the FFT buckets from one sample to the next. "Onset" will generally be represented by a substantial "jump" in this value.

Google "onset detection" for other ideas.

Improve this answer
$\endgroup$
2
$\begingroup$

Using spectral flux alone may produce false positives for certain noises as well as detect a singing voice.

Singing usually implies a signal content containing a pitch, so you could use a pitch detector or estimator (cepstrum, etc.). You could sanity check the fraction of energy that is detected as pitched versus the total signal energy, and that the estimated pitch is within the human voice range. That would reduce the false positive rate for unpitched noise as well as musical sounds outside the normal vocal range.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.