1
$\begingroup$

I am building an application that would "listen" to the microphone input, analyse it, and compare the analysis to a pre-analysed and pre-classified sound bank (small - maximum 20 sounds). It will then show the user what sound it was.

Now, I have a vague idea on how to implement this. I would like to choose a set of features that would best represent the sounds. The issue is that the sounds in the sound bank could be whatever the user recorded. From strong onsets and short sounds, to long onsets and long sounds.

The current features I'm thinking of are:

  • Spectral Centroid
  • Spectral Flux
  • Spectral Rolloff

What do you think? Would these be sufficient to properly classify the sound? Also, as these features output a single value for specific sound buffer, how would you go about handling the feature vector that represents the whole sound? I am using kNN for classification, and was wondering what's the best way to compare two feature vectors? would cross-correlation be a feasible technique?

Thanks a lot!

P.S I have seen that a similar question was asked here, but it doesn't fully answer my issues.

Improve this question
$\endgroup$
12
  • $\begingroup$ I don't know what accuracy or FP/FA rate you want to achieve, but for variety of sounds (unknown) these features won't be enough. When it comes to kNN analysis then it could be some starting point. As for features, use frame based classification and average of it. $\endgroup$
    – jojeck ♦
    Mar 10, 2015 at 21:28
  • $\begingroup$ Thanks @jojek. What other features would you suggest? Also, when you say average do you mean just take the result from each frame and do a simple average? Sum of all elements in feature vector divided by number of frames analysed? $\endgroup$
    – nevos
    Mar 10, 2015 at 21:46
  • $\begingroup$ If, for example, I add MFFCs to the equation, would that be sufficient? How would I go about comparing MFCCs? AFAIK, MFCC extraction outputs coefficients, I need to compare the coefficients to the rest of the sounds? $\endgroup$
    – nevos
    Mar 10, 2015 at 22:00
  • $\begingroup$ I can't help with telling you what features to use, but a very easy way to tell similarity between two vectors is using the 2 norm(a-b). If you aren't familiar with it, in the most simplistic form it is the euclidean distance between two vectors. This will only work if every feature is equally important, otherwise you might have to weight the vectors before doing the norm $\endgroup$
    – andrew
    Mar 10, 2015 at 22:12
  • $\begingroup$ Thanks @andrew, And what if the vectors are of a different length? $\endgroup$
    – nevos
    Mar 10, 2015 at 22:18

2 Answers 2

Reset to default
2
$\begingroup$

Have a look @ librosa, a simple python library for audio analysis, implementing common features. Here is a great introduction and example notebook.

Improve this answer
$\endgroup$
2
  • $\begingroup$ The example notebook link is broken. $\endgroup$
    – Brannon
    Nov 23, 2020 at 16:56
  • $\begingroup$ Fixed the broken link! $\endgroup$
    – ruoho ruotsi
    Nov 26, 2020 at 3:58
0
$\begingroup$

https://github.com/jsingh811/pyAudioProcessing You can use MFCC's as features using the above library for audio classification tasks and a variety of sklearn classifiers.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.