5
$\begingroup$

Problem of processing speech. Required to determine the phonemes and identify vowels and consonants. Anyone involved in this? Please advise what work on the subject is worth reading?

$\endgroup$
12
$\begingroup$

This is somehow a broad question. An interesting book is Digital Processing of speech signals by Rabiner & Schafer, in which you can find some good explanations to get started. It probably is out of date when looking into more advanced techniques, but I think it's good to understand the main characteristics of speech.

In short, every sound we produce is determined by the disposition of the vocal tract. In particular, if the vocal folds are tense and don't vibrate, the sound is said to be unvoiced. However, if they vibrate, the sound is voiced. You can sense the difference by placing your hand in your neck while you say "Ahhhh" or "Ssssshh". In Spanish, all vowels are voiced and all consonants are unvoiced (this is not totally true, please refer to the comments). I'm not sure about other languages though. You can think about a generator (glottis) which is connected to a filter (vocal tract) in a way that sounds take their characteristic shape. This is in fact the basis for speech production models (such as the simplest).

If you look closely to a speech signal, you can easily find differences between the different phonemes. Voiced phonemes are said to be pseudo-periodic. This means that, if you look at a fragment of the signal which is short enough (about tens of milliseconds long, e.g. 40ms), you can find the superposition of harmonic sine signals. This means that:

  • The signal is periodic with a fundamental frequency $f_0$. This is also known as the pitch.
  • Harmonics will appear with frequencies $2f_0, 3f_0, 4f_0$... which energy will depend on the exact phoneme.

If you plot the spectrum of this speech signal, you can find something like the following (source). enter image description here

This resembles, for example, the harmonic structure of a brass instrument sound.

In contrast, unvoiced signals don't have such a clear harmonic structure. In fact, they have no fundamental frequency, and can be conceived as a noisy signal. You can sing by saying "Ahh", but you can't by saying "Ddd". Unvoiced sounds can be conceived as coloured noise. They usually have less energy than voiced.

enter image description here

Taking these differences into account, we can think about a way to detect them automatically.

Voiced sounds are periodic while unvoiced aren't.

To detect periodicity, you can use methods such as autocorrelation (which has been widely treated in this site).

Voiced sounds have more energy than unvoiced.

You can divide the signal in 20-40ms windows, compute the energy for each one and decide according to a threshold.

Unvoiced sounds are noisy and tend to be prominent in high frequencies, while voiced aren't so.

Therefore, unvoiced will cross with zero more often than voiced. This is called Zero Crossing Rate.

While these simple methods are not too accurate by themselves, are a good place to start from. Furthermore, I am sure you can get good results by combining them. In any case, this post does not substitute a good book, as the one I mentioned before. It's just some kind of introduction to show you some possibilities.

$\endgroup$
  • 2
    $\begingroup$ Is that really true, that all consonants are unvoiced in Spanish? What about b, d, hard g, and v? I was always taught they were the same as in English, but is there a subtlety in dialect that actually makes them unvoiced? $\endgroup$ – Zetta Suro Mar 17 '13 at 22:58
  • $\begingroup$ @ZettaSuro That was probably an inaccurate generalization. Looking back into it, I probably misinterpreted the professor's words. After a bit of research, /b/, /d/ and soft /g/ are known as voiced occlusive, but don't seem like to fit in my definition of voiced when looking at their spectrograms, so I think the stated methods would probably work. Hard /g/ seems to be troublesome though see axa here $\endgroup$ – Serge Mar 18 '13 at 12:49
  • $\begingroup$ @ZettaSuro ...and looks voiced except in terms of ZCR. Strong /r/ seems troublesome too. These methods will probably fail on these consonants, but going into the field of isolated phoneme recognition seems overkill for this question. Thanks for letting me know, I have so much to learn! :) $\endgroup$ – Serge Mar 18 '13 at 12:56
  • $\begingroup$ Ah, interesting. I'm new to speech recognition and am coming from an entirely linguistic perspective, so this is good stuff to know. Thanks for the info! $\endgroup$ – Zetta Suro Mar 18 '13 at 15:51
  • $\begingroup$ "To detect periodicity, you can use methods such as autocorrelation" I think measuring the spectral flatness would provide a more directly useful result? $\endgroup$ – endolith Apr 3 '13 at 13:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.