For a Chinese learning app, we let users record a syllable and we use speech recognition to assess if the pronunciation was correct or not.
Every Chinese syllable can be pronounced with different tones (pitch differentials) that have different meanings. We use Beethoven to detect the pitch from the audio to assess this outside of the speech recognition API.
The challenge is that in Chinese the tone is only pronounced in the vowels of syllable. So Beethoven works well if the user only pronounces a vowel, e.g. "a". But in a syllable such as "san" the the results are clouded by the consonants "s" and "n".
So I'm looking for a way to trim the syllable recording to only the vowel so we can use Beethoven on the vowel only and detect the Chinese tone correctly. I'm also happy to learn if anyone has a better idea on how to tackle this challenge.
We found that both Google Translate and Swift Speech framework are not accurate enough to determine if the tone was correct or not.
Best, Paul
