Speech Recognition - Project Idea

Question

Wondering if someone could offer me any advice or help. Sorry if this question is too broad or anything.

I'm wanting to complete an algorithm in basic Speech analysis. The algorithm will detect whether an inputted sample is either saying yes, or, saying no.

I know that this has already been done, and I got my inspiration for this project from someone who has completed it using Zero-Crossing and in MatLab. His algorithm was as follows:

1. Divide the sound wave into evenly spaced blocks

2. Process each block for important characteristics, such as strength across various frequency ranges, number of zero crossings, and total energy.

3. Using this charateristic vector, attempt to associate each block with a phone, which is the most basic unit of speech, producing a string of phones.

4. Find the word whose model is the most likely match to the string of phones which was produced.

Now this algorithm / approach seems idealistic. But, the first point kind of confuses me. Ok, if my data sample is 4410 then how could I possibly split that into equal size blocks? For example, if the data size was 262144 then I know that multiples of 2 goes into it.

The second question is I want to use a FFT/linear predictive coding over zero-crossing but the FFT algorithm is really complicated to implement, so, would it be better to use a already built library?

My last question is this:

Do you think this project is possible, and, in under a year? I'm not completing it in MatLab, but, in C++. Also, will the above algorithm produce the results that I need? Any help would be really appreciated.

Answer 1

A few notes regarding your approach and detailed questions:

• First, it is very common in audio analysis to split the signal to be analyzed into short overlapping windows, because audio signals are not stationary (their characteristics change over time), so they need to be processed on short segments over which they can be considered stationary (many analyses assume stationarity). Typical segment size is in the 5 .. 20 ms range. You'll be left with an odd-size segment at the end, just pad it with zeros and that's it.
• In speech recognition the most widely features are MFCC. AR coefficients are quite common in older literature too. Zero crossing rate or energy are unlikely to discriminate phonemes in a robust way.

Now, regarding your project...

First, it is not clear if the goal of your project is to arrive at a working system even if you have to reuse existing code for that, or to do it yourself to learn on the road...

If you want to reach the goal without learning anything, there are ready to use speech recognition packages like Sphinx or HTK - the catch is that to understand these tools one actually has to know a lot about speech recognition tech. The setup time (training plans, model topology, grammar) and the concepts can be daunting.

An intermediate level would be to use third party code for signal analysis and feature extraction, and then focus on the recognition itself. You'll find C/C++ MFCC or AR modeling code in audio feature extraction packages like YAAFE or Aubio. You can also use the feature extraction command line utility of HTK (HCopy with the right config file for extraction + HList to get a clear text file).

When designing a speech recognition system, you have to ask yourself these questions:

• What is the vocabulary size (from discriminating 2 words to recognizing natural language)?
• What is the speaker variability (will I have to recognize from the same speaker again and again, or will I have to deal with many different speakers)?
• What kind of training data will I have, in particular, is it possible to collect labelled speech samples from the same voice as the one you'll have to recognize?
• What are the conditions (clear recordings... or voice through a glitchy mobile phone with background street noises)?

You have only answered the first question. If the answer to the other ones are these ideal conditions: the system will only deal with one speaker + will have a corpus of a few recordings of yes/no uttered by this same speaker + there will be no background noise ; the problem is not that hard. And these ideals conditions are not that unrealistic - there are some voice control PC applications which require this kind of prior training, the user needs to pronounce a few occurrences of each command word.

In these ideal conditions, a rustic approach (early 80s tech), but which will work, is DTW. Extract from each utterance in the training corpus (which will be labelled either "yes" or "no") a sequence of feature vectors, and compute the DTW distance between the utterance you need to recognize and those in the training corpus. Use this distance metric and a "k-nearest neighbour" scheme to decide on the label. DTW is archaic compared to a proper acoustic model + word model + language model graph, but for a very simple vocabulary (just a handful of words) it does the job.

Now, if you want something speaker independent (which will work for many speakers, including those for which we don't have a template), you'll have three plans. One is to stay with the same approach and record utterances from a very diverse set of speakers in terms of gender, accent, hoping that it will cover enough ground. But DTW scales very poorly! Another is to use an adaptation scheme - find the transform of your feature vectors that minimize the sum of DTW distance (or the minimum DTW distance) between your utterance and the database (paradoxically, I'm sure you'll find more literature about adaptation in the context of classic HMM-based recognition instead of DTW). The last approach is to forego DTW and do the real thing (phone-level GMM + word + language model).

I won't be as negative as the other members. I used to give a very similar project (digit recognition) to students which had no prior experience with audio (but a semester of signal processing and prior experience with Matlab + some background literature before starting the project), and they completed it in Matlab in two or three 3 hours lab sessions (roughly one for MFCC computation, one for DTW, one for the whole corpus management and k-NN + evaluation routines) with about 100 lines of code in total - so this is not an unimaginable engineering feat requiring a full team of PhDs...

Answer 2

Ok, if my data sample is 4410 then how could I possibly split that into equal size blocks?

I wouldn't get too fixated on that. You could just break it up into chunks of size $N$, and then have an odd-sized block at the end that is whatever is left over.

I am not a speech processing expert by any means, but this doesn't seem like the best way to go about it. Like you said, the core of the speech recognition is the phones, and it doesn't seem likely to me that breaking the signal into fixed blocks is a particularly good way to recognize the phones.

The second question is I want to use a FFT/linear predictive coding over zero-crossing but the FFT algorithm is really complicated to implement, so, would it be better to use a already built library?

Yes. Do NOT build your own FFT routines. There are some very good libraries out there already. FFTW is a very good and fast library. There are others that you can find if you google.

Do you think this project is possible, and, in under a year?

You haven't made it very clear what you are trying to accomplish, so that's hard to say.

What I can say is that voice recognition is a very hard problem, which is why they have only gotten decent at it fairly recently. Even now voice recognition software is far from perfect. Given the questions that you are asking I would say that you do not have the background that you need to succeed at something like this. I would either use a commercial tool or open-source software for the voice recognition, or study a lot more if you really want to do it yourself.

Answer 3

I completely agree with Jim Clay's answer; Speech Recognition is hard, and typically requires years of study to do with any kind of accuracy. To guide you toward a more modern approach than that webpage you have listed in your comments, however, the typical method of doing something like this is to transform the chunks of size N to the Mel-frequency cepstrum domain and save the first K coefficients of the new vector, where K << N. (Example values: N = 512, K = 14)

You can then compare the euclidean distance of an "unknown" chunk (Input from a user speaking into a microphone) against your "known" chunks (preprocessed chunks that you know are either "yes" or "no" that you compare against). This is an example of the K-nearest-neighbors algorithm, however there are many other machine-learning techniques used.

If you understand everything that has been said in this answer and Jim Clays, I would say that building and implementing a system like this should take less than 3 months. (I taught a class on Signal Processing where my students had to do projects very similar to this, but they were Signal Processing Seniors, and it took them 10 weeks to build systems comparable to this one) If you do not understand everything, you need to read up before tackling something like this.