5
$\begingroup$

i am making a smart home automation software and i would like to know if there is any software that can recognize noises such as speech ,playing music,phone ringing etc The exact software i am looking for has already been made by mitsubishi but i cant find the source. A video by mitsubishi explaining what i need http://www.merl.com/areas/SoundRecognition/classifier-on-pda.mpeg also the project's site http://www.merl.com/areas/SoundRecognition/

$\endgroup$
1
$\begingroup$

I don't have any practical experience but I found numerous libraries:

You might need to be familiar with classification algorithms to use make good use of them.

$\endgroup$
5
$\begingroup$

The audio feature extraction libraries given above could be a good start, with some caveats:

  • The features you need for this task are plain MFCCs, so most of those libraries are totally overkill since they also extract features which are more meaningful in a musical context.

  • Some of those libraries have been designed for offline analysis (processing a pre-recorded file rather than an ongoing stream of audio buffers). Your application in home-automation suggests that you want something capable of producing on-the-fly results as ambient audio is recorded.

  • None of those libraries actually address the very problem of recognizing sounds ; they just extract audio features upon which classification can be performed, so you need to post-process their output with machine learning software - using a model previously trained on examples of each class of signals you want to discriminate.

You'll end up having less plumbing to do if you use a speech recognition toolbox like HTK or Sphinx. HTK is generic enough for your task - it offers you both feature extraction and recognition/classification with HMM (a very common machine learning technique not only used for speech recognition but also for audio categorization in video analysis), and has extensions for on the fly recognition.

From what I remember, the MERL system you mention, and pretty much every work they did on audio for multimedia document analysis, is based on HMMs on the MPEG-7 audio descriptors.

$\endgroup$
  • $\begingroup$ it doesnt need to be on-the-fly results. I need first to record and then to analyze to find noise patterns that have something to do with other sensor information.As for the training of models the software will ask the home user(on his android) what does a sound mean if it happens frequently and it has connections with other sensor data. $\endgroup$ – vkefallinos May 25 '12 at 4:35
0
$\begingroup$

Essentia in conjunction with Gaia library are able to extract audio features, train classifier models and apply them to recognize different types of sounds. In particular they are used for music classification into genres and moods, and it will be surely possible to build a sound scene classifier.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.