How to Filter Human Speech from an Audio clip?

Question

I'm looking for an open source library to detect Human speech from an audio clip, such as a wav file.

Answer 1

This is not a complete answer, but detection of words from the speech itself is almost always dictionary based - hence one needs to know what words would sound like. If you don't know language, you don't quite know the relation to the sounds. Hence, speech/audio recognition don't quite work.

There can be many primary features to classify:
Reference: Automatic Language Identification: A Review/Tutorial by Yeshwant K. Muthusamy , Etienne Barnard , Ronald A. Cole Signal Processing Magazine, IEEE Oct. 1994 Volume: 11, Issue: 4 pp. 33 - 41

Or related paper of same author.

1. Phonetics:
   Here you should try to look for is that you should try to collect and identify sounds based on primary phonetics. For example - you can use something like International Phonetic Alphabet

2. Prosodic:
   Languages like English, French, German might have very close phonetics. In such case, you can also use Prosodic information which is essentially duration of sound, emphasis or accentuations, pitch and pitch variations.

3. Phonotactics:
   Phonotactics refers to the rules that govern the combinations of the di erent phones in a language. There is a wide variance in phonotactic rules across languages. For example, the phone cluster /sr/ is very common in the Dravidian language Tamil, whereas it is not a legal cluster in English.

4. Vocabulary:
   If you are successful at this, you can probably make another classification of which phonetic alphabet orders are most likely in particular language. You probably don't have to recognize each word. But based on most likely phonetic symbol sequences, you can probably do language detection.

Answer 2

As has been mentioned in the comments, you can't just get a transcript without knowing what language is being spoken. However, you can decode the audio assuming it is of a given language. Thus I would propose the following system:

You run your speech recognizer N times on your audio, where N is the number of languages you are identifying, using the respective language model and acoustic model of each language. This will give you N transcripts of the audio, each based on the assumption that the audio is in that language. However, more significantly, HMM-based speech recognizers (by far the most common type) will also give you an estimate of the log probability of a given utterance, essentially telling you how well the given utterance matches your language and acoustic models. Once you have the log probabilities for each of the different languages, you just need to do a comparison and pick the best one.

Of course, depending on the differences between the models, there might be some bias in these probabilities. For example, the English model give higher probabilities than the Chinese model for almost all input utterances, even if they are in Chinese. To fix this, you need to renormalize these values to ranges where they can be compared. This can only be done by looking at the log probabilities given for large numbers of utterances across a variety of languages, and seeing how they compare between your different languages. You could even see this as a machine learning problem if you wanted to.

I would suggest using CMU Sphinx, an open source recognizer with good performance that has prebuilt language and acoustic models for Enlish, Chinese, French, Spanish, German, Russian, and other lagnuages. Also, I can confirm that at least for the pocketsphinx recognizer, it will output the log probabilities of hypotheses. (In fact, it will even give you the probabilities of the scores of the best few hypotheses for a given model, additional information that might allow you to guess the language even more accurately).