Replicating Research: Feature Vectors of Audio Samples

Question

I'm trying to make a few classifiers with scikitlearn. I'm using the Urban Sound database, and trying to replicated their research for a baseline. However, I don't understand how they derive their feature vector? (I've tried to contact them to no avail).

Here is the passage:

In all experiments we extract the features on a per-frame basis using a window size of 23.2 ms and 50% frame overlap. We compute 40 Mel bands between 0 and 22050 Hz and keep the first 25 MFCC coefficients (we do not apply any pre-emphasis nor liftering). The per-frame values for each coefficient are summarized across time using the following summary statistics: minimum, maximum, median, mean, variance, skewness, kurtosis and the mean and variance of the first and second derivatives, resulting in a feature vector of dimension 275 per slice.

So, 11 metrics * 25 MFCC coefficients == 275 features. I have several concerns:

• This passage seems to use the word "coefficient" to refer to a vector of coefficients, which I thought was the cepstrum (itself being composed of coefficients which are scalar).

• mean, skewness etc are taken on a vector of coefficients (the time series) and return a scalar. But the delta and delta delta return a vector

• How do you get the first 25 coefficients for a frame? Compute N and throw away N - 25? How many should N be?

• If you have 11 * 25 features/frame (since "The per-frame values for each coefficient are summarized across time ...") then wouldn't you end up with many many more features/slice? Depending on the value of frames/slice? But the article says "resulting in a feature vector of dimension 275 per slice."

The dimensionality is really confusing me here..

Answer 1

Here's my interpretation, and I'll summarize using Python's Librosa module:

Each frame is n_fft=512 samples (23.2 ms at sr=22050 Hz sampling rate), and hop_length=256 samples to get 50% overlap. Let y be the audio signal vector. When you run

mfcc_mat = librosa.feature.mfcc(y=y, sr=sr, n_fft=512, hop_length=256, n_mfcc=40)

you'll get a matrix with 40 rows ("coefficients") and $\approx$len(y)/hop_length columns ("time slices"). The authors only use the first 25 rows, and collapse the time dimension in 11 different ways (min, max, median, mean, etc.). Here's the (incomplete) code snippet that generates the 275-length feature vector:

metric_min = np.min( mfcc_mat[0:25,:], axis=1)
metric_max = np.max( mfcc_mat[0:25,:], axis=1)
metric_median = np.median( mfcc_mat[0:25,:], axis=1)
...
feature_vector = np.hstack((metric_min, metric_max, metric_median,...))

etc.

https://librosa.github.io/librosa/generated/librosa.feature.mfcc.html https://librosa.github.io/librosa/generated/librosa.feature.melspectrogram.html#librosa.feature.melspectrogram