Normalize Fourier Transform For Video Game

Question

I am in the process of developing a video game where I would like to sync a character's mouth with an audio clip of their speech that is playing. I found this great website that shows the profiles of different vowel sounds in a frequency spectrum: http://hyperphysics.phy-astr.gsu.edu/hbase/music/vowel.html

After a little (actually, a lot of) effort, I am able to show in real time the spectrum of the audio that is playing.

The next step I am planning to do is find the 4 largest peaks and try to match it to one of the vowel profiles from the website above. However, I have noticed that the relative power of the frequency domain coefficients is always strong lower than higher. The picture I showed is in silence. During an SSSSSS sound, the spectrum does show peaks at higher frequencies:

If I ignore the first few points in the silence FT and then relatively graph the rest, the spectrum has the same shape. If I ignore a few more, again the lower frequencies are much stronger than the higher frequencies.

I assume that this is the result of the math which I can barely get through to code the basic FFT. Should I be normalizing the result somehow to be able to see peaks in the higher frequencies? What process should I use for this?

Thanks in advance for any help you can provide!

---

To add info requested in the comments, by "in real time", I mean that every frame (at about 60 frames per seconds), I get a set of 1024 samples (sampled at 44100 Hz) that is centered at the point of audio that is currently playing). I then compute the FT of these samples and draw the spectrum.

Answer 1

Not sure if your approach is the right path. Anyhow, hearing aid research has produced many "long term average speech spectrum" (LTASS) curves that you can use to whiten the spectrum by dividing it with the LTASS. Have a look at http://www.dydaktyka.cba.pl/AvSpeech.pdf (Byrne et al. 1994 "An international comparison of long‐term average speech spectra" J. Acoust. Soc. Am. 96, 2108) Just don't divide by the LTASS dB values but linearize them first to $10^{LTASS\_in\_dB/20}$ if you are dealing with magnitude frequency spectrum or to $10^{LTASS\_in\_dB/10}$ if you are dealing with power spectrum.

Here's the worldwide average LTASS from Byrne et al. 1994:

Freq.   Male    Female  Combined
(Hz)    (dB)    (dB)    (dB)
63      38.6    37.0    38.6
80      43.5    36.0    43.5
100     54.4    37.5    54.4
125     57.7    40.1    57.7
160     56.8    53.4    56.8
200     58.2    62.2    60.2
250     59.7    60.9    60.3
315     60.0    58.1    59.0
400     62.4    61.7    62.1
500     62.6    61.7    62.1
630     60.6    60.4    60.5
800     55.7    58.0    56.8
1000    53.1    54.3    53.7
1250    53.7    52.3    53.0
1600    52.3    51.7    52.0
2000    48.7    48.8    48.7
2500    48.9    47.3    48.1
3150    47.0    46.7    46.8
4000    46.0    45.3    45.6
5000    44.4    44.6    44.5
6300    43.3    45.2    44.3
8000    42.4    44.9    43.7
10000   41.9    45.0    43.4
12500   39.8    42.8    41.3
16000   40.4    41.1    40.7