5
$\begingroup$

I'm creating an app (I'm a software developer) that for different sounds (interjections from the user) will do different things. For example if the user says:
"Tiu", than it will do task A,
if it will say "Tou" than it will do task B.

In digital sound, what's the difference between sound (user interjection) "Tiu" and "Tou"?

I need a fast way to say when the user said "Tiu" and when it said "Tou", I don't want to use word/speech recognition because it is too slow for my needs.

I'm mainly interested in the theoretical part of digital sounds. E.g. how can I put in the "Tiu" and "Tou" the following sounds "Ti Tiau Tiuu Tiiiu Too Toou Taou", what are their common "features".

(1) edit

After reading the answers, I must confess that I'm new to speech recognition and any similar domain. I didn't new in what I'm getting myself with this app. But overall the research is fun because I love algorithms and math.

Where I am now?

Currently I am reading/researching/learning about CMUSphinx Open Source Toolkit For Speech Recognition http://cmusphinx.sourceforge.net/wiki/tutorial. I don't think that I will use this toolkit but it has a good explanation of the basis in Speech Recognition: Basics of Speech.

Also after reading the responses, I ended to the conclusion that a training phase might be quite helpful, anyway I'm investigating more on it.

New Question:

Do you now any good reading material that might help me in gaining more knowledge in this particular domain?

Can the recording device help me in identifying "tou" and "tiu"? e.g. if the device supports different recording filters - like low frequency filters etc...?

$\endgroup$
  • 4
    $\begingroup$ Forget digital sound, how do the English pronunciations of "tiu" and "tou" differ? $\endgroup$ – Jason R Feb 7 '12 at 22:11
  • 1
    $\begingroup$ makes sense, I edited the question. Thanks for the suggestion :) $\endgroup$ – Stel Feb 7 '12 at 22:18
  • 1
    $\begingroup$ I'm not sure that that's what @JasonR meant. $\endgroup$ – Phonon Feb 7 '12 at 22:44
  • $\begingroup$ @JasonR: I guess we must assume the given inputs "tiu" and "tou" are given already in phonetic transcription--hence only the center vowels differ. The answer will come from a speech processing specialist who may tell us what is the good parameterization of the signal (e.g. formants) for the task and what to look for in the temporal evolution of these speech parameters that would differentiate "tiu" from "tou." $\endgroup$ – Matthias Odisio Feb 7 '12 at 23:59
  • 1
    $\begingroup$ This looks like a gigantic R&D project in speech processing, not an app development project. Especially, if you're trying to beat Android's built-in speech-to-text capabilities. $\endgroup$ – Phonon Feb 8 '12 at 22:23
5
$\begingroup$

First of all, don't expect an answer like "Compute such and such well-known function on the data and if the result is above such or such value it's tiuuu otherwise it's tou". You won't get anything robust or reliable with approaches like that.

The good representations of sound which can be used for this kind of discrimination task are either the Mel Frequency Cepstral Coefficients (MFCC) or the linear prediction coefficients (Compute the first 20 or so autocorrelation coefficients and use the Levinson-Durbin recursion to get the AR coefficients). Both are somehow independent of pitch (good), moderately robust to noise (good but can be better), but are still somewhat speaker and gender dependent. It's still a vast improvement over trying to guess formants from a spectrum and matching that to a per-phoneme table.

Does your application tolerate a training phase? If so, get your user to record a dozen of utterances of each word ; and use this as a training set for a simple bayesian classifier using a gaussian model. Use this classifier to discriminate new utterances.

Does your application need realtime processing? If not, you can get by with clustering. Record the whole "sentence", split it into "words" ; compute average feature vectors over each word ; and run K-means clustering to get 2 class out of it. One class will be "tou" and the other "tiu". The class whose "words" have the lowest average spectral centroid are likely to be the "tou".

A last word on speech recognition: if you have a vocabulary of two words, the cost of running a decent speech recognition toolchain will be that of: * Computing the MFCC features (which you have to do anyway no matter what). * Scoring a dozen of gaussians (which you have to do anyway). * Finding the lowest cost paths in a FST with less than a dozen states.

This is quite pedestrian and nowhere near the complexity of natural language speech recognition. Since you have no experience in digital signal processing, doing so will probably be faster than whatever custom code you might write. Thus I very strongly suggest you to collect a small training corpus of utterances from people around you and start looking into "off the shelf" speech recognition toolboxes like HTK. Just using the built-in tools you will be able to train a model evaluate it on your data without writing code.

$\endgroup$
  • $\begingroup$ Thank You for your answer is very helpful. to answer some of your questions: Right now I don't know exactly I will do the app, e.g. if it will have or not a training phase. I only wanted to have some interjections that can trigger some basic functionality in the app. $\endgroup$ – Stel Feb 12 '12 at 12:28
4
$\begingroup$

Since the differences are only in the vowels, formant detection should do the trick. A vowel typically has periodic excitation, i.e you have a fundamental and harmonic frequencies. The sound gets generated in your throat by the vocal chords (glottis). Your vocal tract (mouth, lips, tongue, upper throat, nasal cavities etc.) create specific resonances that control the relative strength of the harmonics. By moving these parts about you change the acoustic resonance and hence relative harmonic strength. That is then perceived as different vowels. If you stand in front of a mirror and say slowly AAAEEEEIIIIUUUUOOOO you will see exactly what I mean (it's fun to watch:-)).

The vocal spectrum has typically two local peaks in it. These are called "formants". The frequency of the first and second formant defines the vowel. This can be shown in a 2-dimensional graph referred to as formant map. Here is an example.

http://www.phy.davidson.edu/fachome/dmb/digitalspeech/formantmap.gif

Once you've figured out the frequencies of the two formants you can look up the map and find the vowel (nor no-mans land if you are out of luck).

The sounds in your examples have multiple vowels so you need to track the positions of the formants over time and you will get a line on the map (not just a single point). Formant detection is by no means a trivial exercise and many a PhD thesis has been written on the topic. The best approach depends on your application (one speaker vs. multiple speaker, is there a training phase, how big is the vocabulary, how clean is your pickup, etc.)

Formant maps are also highly language and dialect dependent. In fact that's the of the main differences between for example Irish, British, Scottish, Australian, and various US flavors of English.

$\endgroup$
  • $\begingroup$ Thank you for your answer, any info about speech is very helpful! I'm very new to the domain and I had no idea in what I'm getting myself, but is so fun since I love algorithms and math. $\endgroup$ – Stel Feb 12 '12 at 12:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.