I am doing some research related to real-time voice/hotword recognition engines. In most current implementations, input is divided into frames (overlapping or not), and audio features are extracted per frame (most common being MFCCs) and fed into a Hidden Markov Model or a Neural Network of sorts.

Most papers I read address issues such as noise removal/reduction (using methods like Cepstral Mean Normalization), however I couldn't find any mention of how different voice amplitudes are handled. For example, I can train an engine to recognize my voice when I speak normally, however if I change my volume (speak louder), then the extracted features would look different (same shape but larger magnitude). Since this is a real-time system, I am not sure how real-time normalization is even possible, or whether it should be applied on the voice samples, or the extracted features. Or perhaps it is solved by training the system on variations in the speaker's volume?

Your help is greatly appreciated.


Some of the channel effects can be indeed removed by doing the Cepstral Mean Subtraction/Normalization. Nevertheless that generally applies only to "convolutive" distortions that are constant. Any additive distortions, i.e. white noise, babble noise usually cannot be removed via CMS. But like you said this topic is handled via different methods to cope with the noise removal.

As for the immunity of MFCC's with respect to different audio levels, you can do it easily by taking the appropriate coefficients. Generally the first MFCC coefficients is obtained by fitting the constant value curve ($\cos(0)$) to your log-energy filter banks. Therefore it is highly correlated to the RMS energy of your signal. If your remove that coefficient (often called 'static') then in theory you make your model volume (gain) independent. The rest of coefficients is not really related to the energy of your signal. Usually researchers drop the first coefficient, but they are adding it first and second derivative to the $\Delta$ and $\Delta\Delta$ features.

Obviously there is more to that. For example the Lombard Effect might take place and it's obviously changing the envelope of your signal.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.