I have audio samples which MFCCs i want to train, but there is a problem. I can't find a classification model, because the samples have different length and consequently the MFCC matrices will also have different lengths. My question is - which machine learning model can i use?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
When utilizing MFCCs, it is common to split the MFCC matrix into pieces. Lets assume your MFCC-matrix $\bf{M}$ has the dimensions $[\bf{M}]$$=N \times 39$, where $N$ denotes your time index and you have 39 MFCC bins. Then you cut your matrix into blocks of length $B$ (I propose $B=39$ so your blocks have a square shape) and save them in a list. For the last block, you can either zero-pad until it has the same length or discard it completely. Each block has to be annotated with the label of the original audio sample.
A standard approach for classification is a Deep Neural Network, that consists of Convolutional and adjacent Pooling layer since MFCCs are like images locally connected/correlated and Convolutional layers take care of that. Of course, you should not forget to add an MLP layer at the end with a softmax activation to get an estimated probability distribution as an output of your net as well as an appropriate regularization technique.
A more advanced approach would be the use of a recurrent network structure like an LSTM-gate or a plain Recurrent layer. Therefor, you have to store the blocks of a sample in chronological order so the network can learn the temporal structure of a whole sample.
EDIT: Actually one often takes only 13 MFCC bins. The $N \times 39$ matrix also contains the Deltas and DeltaDeltas of the MFCC bins.
-
$\begingroup$ As I understood, i need to collect for example first 13 lists and keep it as a 13X13 matrix and do the same with the other lists as in this scheme - mega.nz/#!l0IFzQab!cUVE-61soHK1UHI5-QRxe2kTHOhsuezBPvbOP13YbYk So at least I will have a dataset that consists of my collected 13X13 matrixes. How should I prepare this matrixes as training data ? $\endgroup$ Jun 7, 2019 at 22:56
-
1$\begingroup$ I assume you are using some package in Python: keras, tensorflow etc.. If that's the case, you would fill an array with the dimensions [amount_samples, 13, 13, 1] with the 13x13 MFCC samples. The last dimension is simply added because the convolutional net requires a parameter for the amount of channels (like in image processing). Since mfccs have only one channel and are thus matrices, we write 1. $\endgroup$– ulfgarJun 10, 2019 at 19:29
-
-
$\begingroup$ Hello for me I used mfcc and other features to feed my cnn and lstm also. In features mfcc extraction, for every audio file I've got 13xN matrix. Then I take the np.mean() of these. So that I finish with a vector of 13x1 is this a good approch. all work I see are donne like that. $\endgroup$– abdoulsnNov 17, 2019 at 16:38