# MFCC classification model

I have audio samples which MFCCs i want to train, but there is a problem. I can't find a classification model, because the samples have different length and consequently the MFCC matrices will also have different lengths. My question is - which machine learning model can i use?

## 1 Answer

When utilizing MFCCs, it is common to split the MFCC matrix into pieces. Lets assume your MFCC-matrix $$\bf{M}$$ has the dimensions $$[\bf{M}]=N \times 39$$, where $$N$$ denotes your time index and you have 39 MFCC bins. Then you cut your matrix into blocks of length $$B$$ (I propose $$B=39$$ so your blocks have a square shape) and save them in a list. For the last block, you can either zero-pad until it has the same length or discard it completely. Each block has to be annotated with the label of the original audio sample.

A standard approach for classification is a Deep Neural Network, that consists of Convolutional and adjacent Pooling layer since MFCCs are like images locally connected/correlated and Convolutional layers take care of that. Of course, you should not forget to add an MLP layer at the end with a softmax activation to get an estimated probability distribution as an output of your net as well as an appropriate regularization technique.

A more advanced approach would be the use of a recurrent network structure like an LSTM-gate or a plain Recurrent layer. Therefor, you have to store the blocks of a sample in chronological order so the network can learn the temporal structure of a whole sample.

EDIT: Actually one often takes only 13 MFCC bins. The $$N \times 39$$ matrix also contains the Deltas and DeltaDeltas of the MFCC bins.

• As I understood, i need to collect for example first 13 lists and keep it as a 13X13 matrix and do the same with the other lists as in this scheme - mega.nz/#!l0IFzQab!cUVE-61soHK1UHI5-QRxe2kTHOhsuezBPvbOP13YbYk So at least I will have a dataset that consists of my collected 13X13 matrixes. How should I prepare this matrixes as training data ? – Armen Hayrapetyan Jun 7 '19 at 22:56
• I assume you are using some package in Python: keras, tensorflow etc.. If that's the case, you would fill an array with the dimensions [amount_samples, 13, 13, 1] with the 13x13 MFCC samples. The last dimension is simply added because the convolutional net requires a parameter for the amount of channels (like in image processing). Since mfccs have only one channel and are thus matrices, we write 1. – ulfgar Jun 10 '19 at 19:29
• Thank you for your help! – Armen Hayrapetyan Jul 8 '19 at 12:21
• Hello for me I used mfcc and other features to feed my cnn and lstm also. In features mfcc extraction, for every audio file I've got 13xN matrix. Then I take the np.mean() of these. So that I finish with a vector of 13x1 is this a good approch. all work I see are donne like that. – abdoulsn Nov 17 '19 at 16:38