Detecting phones in a word audio

Question

I am working on speech recognition for Hindi language. My approach is to map speech to text based on phones. For this, first split the speech into the words and then map each word to text by detecting phones. Till now I am able to split the speech audio file into separated words audio files (I used energy levels for this).

Number of different phones is 1023. For each phone, I have extracted MFCC coefficients for every 10ms frame. (So different phones have different number of frames and for each frame I store 38 coefficients. This serves as templates for each phone.) Also for each word in speech, I extract MFCC coefficients for every 10ms frame.

But I am unable to find efficient algorithm to map word into sequence of phones. I have been thinking of two approaches for this:

1. Naive bruteforce approach in which I simply try to match every possible group of frames in word to every available template (using DTW) and then select the closest match. But this is very inefficient method.
2. Split word into phones. Then for each phone select most appropriate template from 1023 available templates (using DTW). But in this approach, I am not getting how to split word into separate phones.

Is there any another approach to solving this problem? Or can it be done efficiently using second approach?

Answer 1

Though acoustic speech features (MFCCs, etc) & lexical words are correlated in daily language, traditionally in speech recognition software, acoustic and linguistic information are assumed independent.

You can see this in popular open-source speech recognition packages like Sphinx3/4 (Java/C using HMMs vs DTW) or Kaldi (C++/neural nets), one must train both an acoustic model & a language model. For the English language, there are a few off-the-shelf models available, but in most domain-specific cases, it is best to train a new one or adapt an existing model. I'm not sure about the state of acoustic/language models for the Hindi language, but here is a paper you may have seen.

I recommend looking for existing Hindi models to use within Sphinx or Kaldi.