I'm new to speech recognition and deep learning and in a learning phase.

I'm trying to follow this paper to learn how to use RNN as posterior probability estimation in an HTK environment. The paper proposes RNN-HMM hybrid system, so for the HMM part I need to use HTK platform.

The problem is that I couldn't even start from anywhere. I have a sample code which uses HMM to recognize digits, but I'm unable to solve at which part should I insert RNN to the code.

If there are any ideas, I would be glad.

The system in the paper is shown as follows: The method of the paper

I have codes in python environment and HMM is applied using HTK. After converting data to MFCC format, I should use RNN, but after using RNN, which steps of HMM should I apply to generate RNN-HMM acoustic system.


It is very huge effort to build RNN-HMM system with HTK. It is easier to use more flexible toolkits like Kaldi, they have RNN-HMM already implemented.

HTK in latest version 3.5 beta already has support for DNN (not RNN but still a neural network), you can grep for HYBRIDHS in sources. For details on DNN-HMM it is better to read HTKBook 3.5 section 3.12 DNN-HMM Systems and Chapter 14 Artificial Neural Networks

You need to modify all places in HModel.c file to introduce your own type of the model. It is not a separate step you could introduce, you have to modify complex interaction between acoustic model and HMM search code.

It is also not straightforward to embed Python code into HTK C code, you might dump RNN scores in a file and read them from C, otherwise, you will have to reimplement your RNN code in C.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.