1
$\begingroup$

I'm a little confused on how frame based high dimensional features such as mfcc's and bfcc's are applied in the classification of musical instruments.

Are statistical measures of the features used e.g. using the standard deviation across the frames used as a feature? Or is each frame of coefficients used as independent features e.g. 2 frames, 13 coefficients --> 26 features? a combination of both or something completely different?

Thanks

$\endgroup$

1 Answer 1

1
$\begingroup$

Let us say you have a set of songs you want to classify by genre.

There are two approaches if the classification algorithm you use works on unidimensional vectors (eg: neural networks, SVM...):

  • Either you have one feature vector per song; and your feature vector contains statistics (mean, standard deviation, etc.) summarizing all the MFCCs computed at the frame level. In this case, your training set contains one vector per song; and during classification, the classification algorithm is run once per song.
  • Or you have one feature vector per frame. In this case, your training set is made of frames; and the classification algorithm is run once per frame. Voting is used to select the label for the whole songs. It is common to weight the votes to give more important to the middle of the signal (for example, for musical genre classification, the song might start with a slow/soft intro which might bias the classification).

A particularity of both solutions (the so-called "bag of frames" representation) is that they are invariant to permutations in the sequence of MFCC vectors - they do not take into account the temporal structure of the signal. Another approach is to use machine learning algorithms or models (such as HMMs) which can intrinsically deal with time-series - in which case your representation is the sequence of feature vectors.

The approach you describe in which MFCC vectors for each frame are concatenated to yield a gigantic feature vector is not practical: the large feature vectors would make the learning task computationally extensive, it would yield feature vectors of different dimensions for signals of different durations; and two signals identical modulo a delay would have very different feature vectors.

$\endgroup$
1
  • $\begingroup$ Thanks for the response, it's clearer now. In my case I'm looking at drum classification so I think using the statistics may yield better results as frames at the tail of the drum hit may cause issue. Alternatively I could weight the frames by RMS or spectral flatness. Thanks $\endgroup$
    – melinnde
    Jul 22, 2013 at 16:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.