How to deal with audio with different length in Audio Classification

Question

Currently, I am trying to work with the Dataset UrbanSound8K to try some Audio classification. And I got stuck in the preprocessing step already.

Since the audios are of different lengths, like 4 seconds or 0.3 seconds, I found it impossible to directly pass into the whitening algorithms like PCA even after Feature Extraction, using mel-spectrogram/ MFCC.

So my question is what I can do under such circumstance. I was wondering about zero-padding at the end of the shorter sequence. But it seems not working and not going to yield a nice result.

I saw some people using MFCC and summarizing the MFCCs along the time-axis, like mean, variance, kurtosis, skewness.... I think that would work in this case but I just wonder if there are any other ways to do so.

Answer 1

I suggest that you compare things that are similar and don't compare things that are not. You could proceed with a preliminary classification of "short", "medium", and "long" and do a second set of classifications afterward. Within the duration classes, perform as much stretching and padding you need.

Another possibility is to break up everything into the shortest segments, extract short duration features and then uses those as tokens in a sequence.

Audio classification has many pathologies such as having more than one class present in a window.

Answer 2

If the signals were of the same duration but obtained using different sampling rates, so that they have different sample lengths, you can resample the shorter sequence with a higher sampling rate to match that of the longer sequence.

Answer 3

Zero-padding the beginning and/or end of your data is a good approach that does not induce irrelevant information to your original data. One effective choice is going into the frequency domain. You can apply FFT to your time series and consider first m FFT coefficients as your new data. It is a fair assumption because higher frequencies are usually attributed to environmental or measurement noise and the main part of your data is preserved in lower frequency coefficients.

Answer 4

I think you could try "normalizing" and "discretizing" the time variable: given the frequency amplitudes for a generic instant "time", you calculate norm_time = time/max_time where max_time is the length of your input signal (in seconds) and then you calculate the corresponding bin: time_bin=int(norm_time*n_time_bins). Notice, that you introduce a new parameter n_time_bins, which is the number of non-overlapping consecutive subintervals in the input signal time interval. In fact, you just divide the signal time interval of each sound sample in n_time_bins, and then summarize (integrate - sum) the amplitude of each frequency level within each such time subintervals. Denoting by n_freq the number of frequency intervals (class, bins) considered, your feature set for each sample has always n_freq * n_time_bins elements.

Answer 5

Maybe you could try training an encoder-decoder LSTM neural network.

For example

• https://arxiv.org/abs/1603.00982
• https://arxiv.org/abs/1803.08976

From a quick glance at the abstracts, it looks like these authors were more focused on capturing phonosemantic information, but perhaps you could adapt these architectures to your task.