Feature vector for MFCCs

Question

I am working on a project, for which I am finding the onset times of an audio and then extracting the MFCC coefficients for all the times.

Thus if I have n onset times, I would have n feature vectors, corresponding to the MFCC coefficients at those points.

Now I want to characterize a "bol" of the tabla by the the waveform in between the start of one onset and the start of the next onset. This would essentially mean that for one onset time, I would have to take the MFCCs feature vectors at multiple points, and I would have a matrix of onset values for each onset point, instead of just a single vector at each point. Also, the matrix wouldnot always be of the same size, since the gap between 2 onsets might not always be the same.

How do I combine these multiple MFCC vectors to one, or how to I use this matrix for machine learning?

Answer 1

If I understand the problem well, you have a segment of audio between two onsets, for example between an onset at t=2.0s and an onset at t=2.8s, and your MFCC function returns many MFCC vectors - corresponding to small analysis windows between these two instants (In our example, 80 vectors if your MFCC function is set for a rate of 100 analysis frames per second).

But you want a single vector "summarizing" the MFCC for the entire region between the two onsets.

Option 1:

Increase the window size / decrease the frame rate of your MFCC routine so that it yields fewer vectors.

Option 2:

Build a feature vector with the mean and standard deviation of all the MFCC vectors collected within your time segment.

In both cases, set an upper bound on the size of the segment analyzed by such a process - I'd say at most 200ms. For example, if you have an onset at t=5.0s and the next onset is at t=7.0s, the percussive event is likely to be located within the first couple of hundred ms (say between t=5.0s and t=5.5s) - the rest being silence or background noise.