# In speech recognition applications, how are different voice amplitudes handled? (i.e loud vs. quiet tones)

I am doing some research related to real-time voice/hotword recognition engines. In most current implementations, input is divided into frames (overlapping or not), and audio features are extracted per frame (most common being MFCCs) and fed into a Hidden Markov Model or a Neural Network of sorts.

Most papers I read address issues such as noise removal/reduction (using methods like Cepstral Mean Normalization), however I couldn't find any mention of how different voice amplitudes are handled. For example, I can train an engine to recognize my voice when I speak normally, however if I change my volume (speak louder), then the extracted features would look different (same shape but larger magnitude). Since this is a real-time system, I am not sure how real-time normalization is even possible, or whether it should be applied on the voice samples, or the extracted features. Or perhaps it is solved by training the system on variations in the speaker's volume?

As for the immunity of MFCC's with respect to different audio levels, you can do it easily by taking the appropriate coefficients. Generally the first MFCC coefficients is obtained by fitting the constant value curve ($\cos(0)$) to your log-energy filter banks. Therefore it is highly correlated to the RMS energy of your signal. If your remove that coefficient (often called 'static') then in theory you make your model volume (gain) independent. The rest of coefficients is not really related to the energy of your signal. Usually researchers drop the first coefficient, but they are adding it first and second derivative to the $\Delta$ and $\Delta\Delta$ features.