1
$\begingroup$

I'm looking for a realtime VAD implementation like Discord/Mumble/Teamspeak have.

My goal is to have a system that let's me use the same VAD for all my VoIP applications, as well as something I can use to facilitate VAD for push to talk only applications.

Mumble is open source though I wasn't able to find what I needed when I had a look.

$\endgroup$
0
$\begingroup$

So, this is not an answer in the quality I'd like it to be, but the Mumble VAD is in AudioInput.c , as a quick browsing yielded:

speex_preprocess_ctl(sppPreprocess, SPEEX_PREPROCESS_GET_PROB, &prob);
fSpeechProb = static_cast<float>(prob) / 100.0f;

// clean microphone level: peak of filtered signal attenuated by AGC gain
dPeakCleanMic = qMax(dPeakSignal - gainValue, -96.0f);
float level = (g.s.vsVAD == Settings::SignalToNoise) ? fSpeechProb : (1.0f + dPeakCleanMic / 96.0f);

bool bIsSpeech = false;

if (level > g.s.fVADmax)
    bIsSpeech = true;

What's done here is that Speex, a FOSS voice codec and utility pack is used to get an estimate for the silence->voice transition. Whyever, Mumble converts that probability value (probabilities: $\in[0,1]$) to a percentage, and then compares it to the internal setting.

So the algorithm of interest here is definitely not within Mumble, but libspeexdsp!

Funnily, that's basically where it goes down in speexdsp's preprocess.dsp

/* FIXME: This VAD is a kludge */
st->speech_prob = Pframe;
if (st->vad_enabled)
{
    if (st->speech_prob > st->speech_prob_start || (st->was_speech && st->speech_prob > st->speech_prob_continue))
    {
        st->was_speech=1;
        return 1;
    } else
    {
        st->was_speech=0;
        return 0;
    }
} else {
    return 1;
}

So, this really just boils down to an SNR estimate of the current audio frame:

/* Speech probability of presence for the entire frame is based on the average filterbank a priori SNR */
Zframe = 0;
for (i=N;i<N+M;i++)
    Zframe = ADD32(Zframe, EXTEND32(st->zeta[i]));
Pframe = QCONST16(.1f,15)+MULT16_16_Q15(QCONST16(.899f,15),qcurve(DIV32_16(Zframe,st->nbands)));
$\endgroup$
  • $\begingroup$ I'm trying to get this working. Had trouble getting speexdsp installed on windows so I set up a VM with a linux distro. How do I read in audio from the microphone to pass to the speexdsp library? $\endgroup$ – Tom Feb 21 '17 at 14:28
  • $\begingroup$ speex.org/docs/manual/speex-manual/node13.html $\endgroup$ – Marcus Müller Feb 21 '17 at 14:36
  • $\begingroup$ I've seen that, but I still can't work out how to pass my microphone as an input. I was hoping that would just be as simple as say /dev/mic/, but it doesn't seem to be. $\endgroup$ – Tom Feb 22 '17 at 6:10
  • $\begingroup$ I might be getting somewhere with PortAudio. I've got a data stream from my microphone, though I'm not confident it is the format speexdsp will need. I'll report back with more info if I work it out. $\endgroup$ – Tom Feb 24 '17 at 13:08
  • $\begingroup$ How does this fare on discriminating speech from music? $\endgroup$ – Petrus Theron Aug 27 '18 at 23:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.