I am in the process of developing a video game where I would like to sync a character's mouth with an audio clip of their speech that is playing. I found this great website that shows the profiles of different vowel sounds in a frequency spectrum: http://hyperphysics.phy-astr.gsu.edu/hbase/music/vowel.html
After a little (actually, a lot of) effort, I am able to show in real time the spectrum of the audio that is playing.
The next step I am planning to do is find the 4 largest peaks and try to match it to one of the vowel profiles from the website above. However, I have noticed that the relative power of the frequency domain coefficients is always strong lower than higher. The picture I showed is in silence. During an SSSSSS sound, the spectrum does show peaks at higher frequencies:
If I ignore the first few points in the silence FT and then relatively graph the rest, the spectrum has the same shape. If I ignore a few more, again the lower frequencies are much stronger than the higher frequencies.
I assume that this is the result of the math which I can barely get through to code the basic FFT. Should I be normalizing the result somehow to be able to see peaks in the higher frequencies? What process should I use for this?
Thanks in advance for any help you can provide!
To add info requested in the comments, by "in real time", I mean that every frame (at about 60 frames per seconds), I get a set of 1024 samples (sampled at 44100 Hz) that is centered at the point of audio that is currently playing). I then compute the FT of these samples and draw the spectrum.