1
$\begingroup$

I am planning to do a self project on speech recognition for a vocabulary detection system using arduino. I know CMUsphinx or bitvoicer is probably the way to go. But I would like to code the project from the ground up as the purpose of the project is learning.

I have been reading different papers like

http://revistaie.ase.ro/content/46/s%20-%20furtuna.pdf (dynamic time warping)

http://citeseer.ist.psu.edu/viewdoc/download;jsessionid=DEC7F40E5AE89F1508A3B6D0625CC1DA?doi=10.1.1.131.2084&rep=rep1&type=pdf (HMM)

But I have encountered the following difficulties

  1. I have a hard time reconciling the entire process with higher level algorithm. From what I have read so far the entire process is the following

    i. input signal

    ii. filter all noise, getting all signals that is high energy volume and low frequency. (How do I know what to compare against? What is consider high energy volume, and low freqency?)

    iii. divide the signal into phones using FFT (where do I get dataset for phones, what constitutes a boundary between two phones, and how similar does the detected phones have to match the dataset?)

    iv. matching the list of phones to each word using HMM or DTW

For beginners like me I feel that steps 2-3 is the hardest part due to the difficulties I listed.

Can someone point me a direction?

Thanks

$\endgroup$
2
$\begingroup$

system using arduino

Arduino performance is not enough for any serious speech recognition. You should better play with your mobile phone instead. With mobile you can build at least some interesting applications. Or try Raspberry Pi.

From what I have read so far the entire process is the following

You probably need to read more papers (including the source code of existing software like CMUSphinx) to understand the details because the description of the process you have is not very accurate either. Ideally you need to read a textbook like "Fundamentals of speech recognition" by L. Rabiner.

i. input signal

This is two lines of code.

ii. filter all noise, getting all signals that is high energy volume and low frequency. (How do I know what to compare against? What is consider high energy volume, and low freqency?)

This is not very accurate description, there is no such thing as "low frequency". There is a voice activity detection (VAD) task which is pretty complex and has many dedicated algorithms. The simple form is VAD based on energy of the signal. You compare background energy level with current energy level and if the difference is higher than threshold you count audio as speech. This is not very accurate VAD but it is ok for many cases (no big noise, clean audio). It can be extended to frequency subbands and mixed with noise suppression by spectral subtraction. Noise suppression combined with VAD is even more complex task to solve.

For simple start you can implement energy-based VAD (google can give you many examples of implementation)

iii. divide the signal into phones using FFT (where do I get dataset for phones, what constitutes a boundary between two phones, and how similar does the detected phones have to match the dataset?)

For DTW comparison with the prerecorded audio you do not need to care about phones at all. For HMM-based recognition, HMM decoding algorithm finds boundaries between phones implicitly but actually it doesn't even need them. Speech is considered as a sequence of subphone states, not as phones. Phones are not very stable to model with HMM. You can find description of the algorithm in the Rabiner's paper linked above.

The database of phone properties is extracted from transcribed recordings. There are many public databases available (some are mentioned on cmusphinx website).

iv. matching the list of phones to each word using HMM or DTW

There is no such thing neither in HMM nor in DTW. You match the audio to the word model, not taking individual phones into account. You will have to implement a separate training algorithm.

For beginners like me I feel that steps 2-3 is the hardest part due to the difficulties I listed.

Since your process is not very accurate the difficulty estimation is not very meaningful too. HMM training and decoding are quite complex tasks, VAD or FFT are simple.

The topic is not very easy, it is complex. You can not expect to implement calculus algorithm without knowing the theory of calculus, like what is integration and so on, so the speech recognition. There are many tutorials around, this one is ok to get started too:

http://kastnerkyle.github.io/blog/2014/05/22/single-speaker-speech-recognition/

Maybe it will be helpful for you. However, take into account that any simple tutorial is usually not very professional and doesn't really cover things. Speech recognition can not really be explained in 10 pages of text. If you really want to understand a topic, read a textbook or attend a course on ASR.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.