Feature extraction for sound classification

Question

I'm trying to extract features from a sound file and classify the sound as belonging to a particular category (eg : dog bark, vehicle engine e.t.c). I'd like some clarity on the following things :

1) Is this doable at all? There are programs that can recognize speech, and differentiate between different types of dog bark. But is it possible to have a program that can receive a sound sample and just say what kind of a sound it is? (Assume there's a database containing a lot of sound samples to refer to). The input sound samples can be a bit noisy (microphone input).

2) I assume that the first step is audio feature extraction. This article suggests extracting MFCCs and feeding them to a machine learning algorithm. Is MFCC enough? Are there any other features that are generally used for sound classification?

Thank you for your time.

Answer 1

1. By long shot it is doable - to what extend? You will see. This task of environmental sound classification is not very well studied. Also choice of machine learning paradigm is crucial - statistical approach or maybe binary classifier? You can start with GMM's, ANN's and SVM's - I opt for GMM's and ANN's.
2. Yes, most of people are using MFCC's because they are well correlated with what people are actually hearing and also no one came up with anything better since. You might also want to add extra features such as MPEG-7 descriptors. Proper feature optimisation must be performed because sometimes you don't need so many features, especially when they are do not separable. For more info please refer to my previous answers:

Feature extraction from spectrum

MFCC extraction

Detection of sounds

Answer 2

Non-verbal Audio (let alone environmental) seems to be the little brother to main stream machine learning media types like images, speech, text.

To answer your question is it possible to train a network to identify a given sound? Yes, it is! But it's hard for all the same reasons machine learning is hard.

However what's really holding Audio back, and why I call it the little brother to images and speech, is because of Audio's lack of a large scale labeled dataset. For Speech there's TIMIT, for Images there are several ImagenNet, CIFAR, Caltech, for Text and Natural Language Processing there are vast volumes of literature, etc.

To my knowledge the largest two non-verbal human* labeled audio datasets are the UrbanSounds and ESC-100 datasets, which are prohibitively small for truly deep learning approaches. There are some published mixed results on these datasets using 2-layer ConvNet's.

MFCC features are a well established baseline feature representation in speech recognition and audio analysis in general. But there are tons of other audio feature representations! This paper gives a nice taxonomy of audio feature types.

The most exciting work doing sound classification I've recently seen is being done by some folks at DeepMind, called WaveNet.

Answer 3

Here is a solution for sound classification for 10 classes: dog barking, car horn, children playing etc. It is based on tensorflow library using neural networks. Features are extracted by converting sound clips to spectrogram

Answer 4

Yes, it's extremely doable. Although NNs are excellent at this kind of classification training, they may not even be necessary -- with a well-chosen set of features, just the classic clustering algorithms such as a Gaussian mixture model, or principal component analysis, would probably do as well. Modern libraries can get this stuff right about 95% of the time or more.