Which audio features should be used to detect tone from audio? Are there any open source frameworks or implementations which can be ported to mobile phones to process audio? Alternatively, would it be possible to collect audio features from the phone and then process this on the server so as to detect tone?
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
I dont know how bound your problem is... Is this dealing only with words that are already detected?
This is a very very hard problem, as @JimClay mentions. For such problems, one cannot simply say "pick highest "x"", because the features are not obvious to us. (Even though they might be obvious to non-observable states within our brains, whose signal processing paradigms are unfortunately inaccessible to us via introspection).
So in problems such as those, I would first collect a large sample of "angry" tones, and a large sample of "happy" tones to contrast. You now have two options:
1) I would 'brute force' it and start looking for features to extract that separate them amicably in feature space. It is honestly hard to say where to begin without looking at real data. VERY coarsely speaking, I imagine angry tones have more low-frequency information, VS higher-pitched happy ones. This then lends itself to including the DFT of the data as 1 feature in DFT_length dimensional space. Of course, not all 'low pitchy' sounds are angry, and not all high-pitchy sounds are happy, so you need another feature vector(s) on which to discriminate, lest you inadvertently create a male-female detector...
2) I would use a different approach and simply label my data vectors, (here, each data vector is the time domain snippet of detected words, all of which are re-sampled so that they have the same pre-determined length). I would then start off with a simple linear perceptron to try and create a hyper-plane to best separate the two classes. (You can add an extra dimension to your classifier to make linearly un-separable clusters separable by taking your hyper-plane up one more dimension). Once adequately trained, you can use that weight vector as your classifier and project all data vectors unto it to determine which class it falls under. Bear in mind you need A LOT of training data for this to work properly, because there are many situations where emotions can manifest itself in words and phrases. This is why I asked you how bound your problem is. (Words? Phrases? Sentences?, etc).
There are a number of papers on this topic in IEEE.
This a thesis on the topic available to download. You can look at the papers in the thesis bibliography.