Tell me more ×
Signal Processing Stack Exchange is a question and answer site for practitioners of the art and science of signal, image and video processing. It's 100% free, no registration required.
up vote 1 down vote favorite

Which audio features should be used to detect tone from audio? Are there any open source frameworks or implementations which can be ported to mobile phones to process audio? Alternatively, would it be possible to collect audio features from the phone and then process this on the server so as to detect tone?

share|improve this question
Could you elaborate on which "tones" you want to detect? Your application is a bit unclear... – pichenettes Dec 2 '12 at 1:51
I'm sorry , i should have explained in greater detail. Tones could be "anger", "happy" and so on. – Dony George Dec 2 '12 at 4:19
2  
I am not an expert on the topic, but I know enough to think that this is likely to be a very hard problem in general, and likely to be entirely intractable unless you limit it to, say, English speakers. The reason is that how your voice changes will almost certainly depend on your language and/or culture. – Jim Clay Dec 3 '12 at 13:17
1  
As @JimClay mentions this is a very very hard problem, so I think your best bet is to use supervised learning instead of brute-force feature extraction. – Mohammad Dec 3 '12 at 14:36
2  
This question has a large non-signal-processing part: How are emotions encoded in spoken language? Maybe accentuation of phonems in a word, or of words in a sentence? Have you looked for psychological experiments on this? Maybe that would give you an idea which features could be useful. – nikie Dec 4 '12 at 8:41

2 Answers

up vote 2 down vote accepted

I dont know how bound your problem is... Is this dealing only with words that are already detected?

This is a very very hard problem, as @JimClay mentions. For such problems, one cannot simply say "pick highest "x"", because the features are not obvious to us. (Even though they might be obvious to non-observable states within our brains, whose signal processing paradigms are unfortunately inaccessible to us via introspection).

So in problems such as those, I would first collect a large sample of "angry" tones, and a large sample of "happy" tones to contrast. You now have two options:

1) I would 'brute force' it and start looking for features to extract that separate them amicably in feature space. It is honestly hard to say where to begin without looking at real data. VERY coarsely speaking, I imagine angry tones have more low-frequency information, VS higher-pitched happy ones. This then lends itself to including the DFT of the data as 1 feature in DFT_length dimensional space. Of course, not all 'low pitchy' sounds are angry, and not all high-pitchy sounds are happy, so you need another feature vector(s) on which to discriminate, lest you inadvertently create a male-female detector...

2) I would use a different approach and simply label my data vectors, (here, each data vector is the time domain snippet of detected words, all of which are re-sampled so that they have the same pre-determined length). I would then start off with a simple linear perceptron to try and create a hyper-plane to best separate the two classes. (You can add an extra dimension to your classifier to make linearly un-separable clusters separable by taking your hyper-plane up one more dimension). Once adequately trained, you can use that weight vector as your classifier and project all data vectors unto it to determine which class it falls under. Bear in mind you need A LOT of training data for this to work properly, because there are many situations where emotions can manifest itself in words and phrases. This is why I asked you how bound your problem is. (Words? Phrases? Sentences?, etc).

share|improve this answer
+1. I am not confident that the approaches you describe would work, but I agree that they have the best chance of success. – Jim Clay Dec 3 '12 at 15:20
@JimClay Neither am I. :-) Its my best shot without looking at actual data. – Mohammad Dec 3 '12 at 15:24
up vote 0 down vote

There are a number of papers on this topic in IEEE.

This a thesis on the topic available to download. You can look at the papers in the thesis bibliography.

share|improve this answer
Is this a Master Thesis, a PhD Thesis? More importantly, is this your thesis (you should mention that if that's true)? Also, answers containing only links to resources without any explanation are generally discouraged: it would be much nicer if you can provide a sentence or two of summary of the basic ideas you're recommending with your literature, or why you think the literature is good. – penelope Dec 4 '12 at 9:04
1  
for why only-link answers are bad, take a look here: meta.stackoverflow.com/a/8259 – penelope Dec 4 '12 at 9:10

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.