I have audio samples which MFCCs i want to train, but there is a problem. I can't find a classification model, because the samples have different length and consequently the MFCC matrices will also have different lengths. My question is - which machine learning model can i use?


1 Answer 1


When utilizing MFCCs, it is common to split the MFCC matrix into pieces. Lets assume your MFCC-matrix $\bf{M}$ has the dimensions $[\bf{M}]$$=N \times 39$, where $N$ denotes your time index and you have 39 MFCC bins. Then you cut your matrix into blocks of length $B$ (I propose $B=39$ so your blocks have a square shape) and save them in a list. For the last block, you can either zero-pad until it has the same length or discard it completely. Each block has to be annotated with the label of the original audio sample.

A standard approach for classification is a Deep Neural Network, that consists of Convolutional and adjacent Pooling layer since MFCCs are like images locally connected/correlated and Convolutional layers take care of that. Of course, you should not forget to add an MLP layer at the end with a softmax activation to get an estimated probability distribution as an output of your net as well as an appropriate regularization technique.

A more advanced approach would be the use of a recurrent network structure like an LSTM-gate or a plain Recurrent layer. Therefor, you have to store the blocks of a sample in chronological order so the network can learn the temporal structure of a whole sample.

EDIT: Actually one often takes only 13 MFCC bins. The $N \times 39$ matrix also contains the Deltas and DeltaDeltas of the MFCC bins.

  • $\begingroup$ As I understood, i need to collect for example first 13 lists and keep it as a 13X13 matrix and do the same with the other lists as in this scheme - mega.nz/#!l0IFzQab!cUVE-61soHK1UHI5-QRxe2kTHOhsuezBPvbOP13YbYk So at least I will have a dataset that consists of my collected 13X13 matrixes. How should I prepare this matrixes as training data ? $\endgroup$ Jun 7, 2019 at 22:56
  • 1
    $\begingroup$ I assume you are using some package in Python: keras, tensorflow etc.. If that's the case, you would fill an array with the dimensions [amount_samples, 13, 13, 1] with the 13x13 MFCC samples. The last dimension is simply added because the convolutional net requires a parameter for the amount of channels (like in image processing). Since mfccs have only one channel and are thus matrices, we write 1. $\endgroup$
    – ulfgar
    Jun 10, 2019 at 19:29
  • $\begingroup$ Thank you for your help! $\endgroup$ Jul 8, 2019 at 12:21
  • $\begingroup$ Hello for me I used mfcc and other features to feed my cnn and lstm also. In features mfcc extraction, for every audio file I've got 13xN matrix. Then I take the np.mean() of these. So that I finish with a vector of 13x1 is this a good approch. all work I see are donne like that. $\endgroup$
    – abdoulsn
    Nov 17, 2019 at 16:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.