Which audio features should be used to detect tone from audio? Are there any open source frameworks or implementations which can be ported to mobile phones to process audio? Alternatively, would it be possible to collect audio features from the phone and then process this on the server so as to detect tone?
5
-
$\begingroup$ Could you elaborate on which "tones" you want to detect? Your application is a bit unclear... $\endgroup$– pichenettesCommented Dec 2, 2012 at 1:51
-
$\begingroup$ I'm sorry , i should have explained in greater detail. Tones could be "anger", "happy" and so on. $\endgroup$– Dony GeorgeCommented Dec 2, 2012 at 4:19
-
2$\begingroup$ I am not an expert on the topic, but I know enough to think that this is likely to be a very hard problem in general, and likely to be entirely intractable unless you limit it to, say, English speakers. The reason is that how your voice changes will almost certainly depend on your language and/or culture. $\endgroup$– Jim ClayCommented Dec 3, 2012 at 13:17
-
1$\begingroup$ As @JimClay mentions this is a very very hard problem, so I think your best bet is to use supervised learning instead of brute-force feature extraction. $\endgroup$– SpaceyCommented Dec 3, 2012 at 14:36
-
2$\begingroup$ This question has a large non-signal-processing part: How are emotions encoded in spoken language? Maybe accentuation of phonems in a word, or of words in a sentence? Have you looked for psychological experiments on this? Maybe that would give you an idea which features could be useful. $\endgroup$– Niki EstnerCommented Dec 4, 2012 at 8:41
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
I dont know how bound your problem is... Is this dealing only with words that are already detected?
This is a very very hard problem, as @JimClay mentions. For such problems, one cannot simply say "pick highest "x"", because the features are not obvious to us. (Even though they might be obvious to non-observable states within our brains, whose signal processing paradigms are unfortunately inaccessible to us via introspection).
So in problems such as those, I would first collect a large sample of "angry" tones, and a large sample of "happy" tones to contrast. You now have two options:
1) I would 'brute force' it and start looking for features to extract that separate them amicably in feature space. It is honestly hard to say where to begin without looking at real data. VERY coarsely speaking, I imagine angry tones have more low-frequency information, VS higher-pitched happy ones. This then lends itself to including the DFT of the data as 1 feature in DFT_length dimensional space. Of course, not all 'low pitchy' sounds are angry, and not all high-pitchy sounds are happy, so you need another feature vector(s) on which to discriminate, lest you inadvertently create a male-female detector...
2) I would use a different approach and simply label my data vectors, (here, each data vector is the time domain snippet of detected words, all of which are re-sampled so that they have the same pre-determined length). I would then start off with a simple linear perceptron to try and create a hyper-plane to best separate the two classes. (You can add an extra dimension to your classifier to make linearly un-separable clusters separable by taking your hyper-plane up one more dimension). Once adequately trained, you can use that weight vector as your classifier and project all data vectors unto it to determine which class it falls under. Bear in mind you need A LOT of training data for this to work properly, because there are many situations where emotions can manifest itself in words and phrases. This is why I asked you how bound your problem is. (Words? Phrases? Sentences?, etc).
-
$\begingroup$ +1. I am not confident that the approaches you describe would work, but I agree that they have the best chance of success. $\endgroup$– Jim ClayCommented Dec 3, 2012 at 15:20
-
$\begingroup$ @JimClay Neither am I. :-) Its my best shot without looking at actual data. $\endgroup$– SpaceyCommented Dec 3, 2012 at 15:24
$\begingroup$
$\endgroup$
2
There are a number of papers on this topic in IEEE.
This a thesis on the topic available to download. You can look at the papers in the thesis bibliography.
-
$\begingroup$ Is this a Master Thesis, a PhD Thesis? More importantly, is this your thesis (you should mention that if that's true)? Also, answers containing only links to resources without any explanation are generally discouraged: it would be much nicer if you can provide a sentence or two of summary of the basic ideas you're recommending with your literature, or why you think the literature is good. $\endgroup$– penelopeCommented Dec 4, 2012 at 9:04
-
1$\begingroup$ for why only-link answers are bad, take a look here: meta.stackexchange.com/a/8259 $\endgroup$– penelopeCommented Dec 4, 2012 at 9:10