i am making a smart home automation software and i would like to know if there is any software that can recognize noises such as speech ,playing music,phone ringing etc The exact software i am looking for has already been made by mitsubishi but i cant find the source. A video by mitsubishi explaining what i need http://www.merl.com/areas/SoundRecognition/classifier-on-pda.mpeg also the project's site http://www.merl.com/areas/SoundRecognition/
$\begingroup$
$\endgroup$
$\begingroup$
$\endgroup$
I don't have any practical experience but I found numerous libraries:
- http://jmir.sourceforge.net/
- http://libxtract.sourceforge.net/
- http://yaafe.sourceforge.net/
- http://feapi.sourceforge.net/
- https://sites.google.com/site/pdescriptors/
- http://clam-project.org/
- http://taps.cs.princeton.edu/
- http://aubio.org/
You might need to be familiar with classification algorithms to use make good use of them.
-
$\begingroup$ thank you for that.most of these libraries are music-oriented.I wanted something that has to do with home automation.I found taps.cs.princeton.edu interesting and will try it. $\endgroup$ – vkefallinos May 25 '12 at 4:37
-
$\begingroup$ I also found HARK winnie.kuis.kyoto-u.ac.jp/HARK very interesting as it is made for a home robot.Its open source and it can do pretty good stuff.check it out youtube.com/watch?v=rhBnLySsmJk $\endgroup$ – vkefallinos May 25 '12 at 4:45
$\begingroup$
$\endgroup$
The audio feature extraction libraries given above could be a good start, with some caveats:
The features you need for this task are plain MFCCs, so most of those libraries are totally overkill since they also extract features which are more meaningful in a musical context.
Some of those libraries have been designed for offline analysis (processing a pre-recorded file rather than an ongoing stream of audio buffers). Your application in home-automation suggests that you want something capable of producing on-the-fly results as ambient audio is recorded.
None of those libraries actually address the very problem of recognizing sounds ; they just extract audio features upon which classification can be performed, so you need to post-process their output with machine learning software - using a model previously trained on examples of each class of signals you want to discriminate.
You'll end up having less plumbing to do if you use a speech recognition toolbox like HTK or Sphinx. HTK is generic enough for your task - it offers you both feature extraction and recognition/classification with HMM (a very common machine learning technique not only used for speech recognition but also for audio categorization in video analysis), and has extensions for on the fly recognition.
From what I remember, the MERL system you mention, and pretty much every work they did on audio for multimedia document analysis, is based on HMMs on the MPEG-7 audio descriptors.
answered May 25 '12 at 0:30
-
$\begingroup$ it doesnt need to be on-the-fly results. I need first to record and then to analyze to find noise patterns that have something to do with other sensor information.As for the training of models the software will ask the home user(on his android) what does a sound mean if it happens frequently and it has connections with other sensor data. $\endgroup$ – vkefallinos May 25 '12 at 4:35
$\begingroup$
$\endgroup$
Essentia in conjunction with Gaia library are able to extract audio features, train classifier models and apply them to recognize different types of sounds. In particular they are used for music classification into genres and moods, and it will be surely possible to build a sound scene classifier.
