I am looking for ANY advice anyone can give me regarding on this problem.

I'm looking to start a project on subtitles, basically, if we take a Youtube video of a breaking news story, subtitles are generated automatically without the need of physically inputting what the speaker is saying. Now, I understand the complexity of this problem, and, I am aware of the amount of training samples I would need in order to carry out such a project.

The question is this: I have had experience in using DTW and HMM in order to train and recognise simple words "Yes", "No" and gained a lot of experience from this and as part of my previous findings I found that the DTW algorithm is slow and therefore would not be efficient for this. For this project, I imagine real-time processing is involved. Would it therefore be a wise thing to implement a Hidden Markov Model for such a problem, or, would another model/algorithm be more suitable?

Here is my basic structure:

I will upload a 3 minute video to youtube, I will then extract the raw audio from this video and then extract the features (MFCC) which will then be used in order to train the model (HMM) which will /hopefully/ identify the spoken word.

Could anyone recommend any papers that could potentially help me with tackling such a problem?

Thank you for taking time to read this. If anything is not understood, please let me know.


The amount of knowledge necessary to develop such a large scale multi-language, speaker-independent, large-vocabulary speech recognition system is spread well over hundreds of papers; and each individual brick of the system (say the feature extraction front-end, the FST decoding library, the language model store) is developed by world-class expert in this specific field. Some of Google's research scientists have published papers on these topics, this might be a good place to start if you want to understand what's behind the scenes... But maybe you should start with a smaller goal - like building a system that has a vocabulary of a dozen different words from the same speaker - using a library like HTK or Sphinx (using the "canonical" approach with the hierarchy GMM->phone->word->language model), and once you have all the basics right, explore deeper in one direction (try more complex feature extraction methods to handle noise, grow the vocabulary and the complexity of the grammar, try speaker adaptation).

  • $\begingroup$ Thanks for your reply, and, your honest feedback. Yes, looking at it from this point of view does suggest that it would be too complex for me (at this current time!) With "Complexity of grammar" in that I recognise whether someone is saying "To" or "Two" might be worth considering as a project? $\endgroup$
    – Phorce
    Jul 1 '13 at 20:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.