Recognizing bodily noises (ie, not spoken words)

Question

I was pointed here by a chain of stackoverflow posts. I'm a software developer looking to make a module that recognizes noises like snorts and coughs. Here was my original question on stackoverflow and here is where I found this SE board.

In short, I'm trying to use a Raspberry Pi and a microphone to pick up noises like snorts, coughs, sneezes, etc as a proof of concept to tell if someone is sick. I'm not quite sure where to start, as those posts I linked above have a lot of information. I've also come across this post, but I'm not sure if that helps.

I'm very new here and to this particular concept, so any and all help would be appreciated.

Answer 1

Most likely you are interested in a very simple approach that will run on your Pi. Some of methods I am going to mention are possible to run in real-time with not-badly written C code. Probably you will start with HMM's and MFCC's - it's not a bad idea for the beginning, but as you will see - people developed more advanced methods for such application. Choice of features is rather obvious, pre-processing and Machine Learning is up to you - this will have biggest impact onto your results.

Here is a list of some papers you might find interesting. All of them are targeting cough recognition problem and are applicable to other types of sounds.

HMM-like approach:

Takahashi S., et al. - Cough Detection in Spoken Dialogue System for Home Health Care

Shasha Le, Weiping Hu - Cough sound recognition based on Hilbert marginal spectrum

Matos S., et al. - Detection of cough signals in continuous audio recordings using HMM's

Chunmei Z., et al. - Recognition of cough using features improved by sub-band energy transformation

Hu W., et al. - Cough sound detection bases on EMD analysis and HMM recognition

Karhunen–Loève-like approach:

Liu Y., et al. - A cough sound recognition algorithm based on time-frequency energy distribution

Decision Trees-like approach:

Larson. E, et al. - Accurate and Privacy Preserving Cough Sensing using a Low-Cost Microphone

Martinek J., et al. - Distinction of cough from other sounds produced by daily activities in the upper airways

SVM-like approach:

Jia-Ming L., et al. - Cough signal recognition with Gammatone Cepstral Coefficients

And this is absolutely lovely ;)

Answer 2

As I've understood, your project is to detect and distinguish sounds like snorts, coughs, etc. from a domestic environment. So you are NOT interested in other sounds, like human speech, or other possible noises there.

If that is right, an ostensible approach is to use a learning scheme along with an efficient set of features to $\bf{classify}$ the input sounds to the system. A function which can tell you whether a particular frame of audio ( fed to the system ) is of the sort that you are looking for or is not.

A promising fact about your project, is that the voices made by the human body are usually pretty distinctive from the background sounds. It is the consequence of the special features of the vocal track of men, which can posses (more or less) unique properties.

So, I suggest that you carry out the following steps:

1. Gather a sufficient database of the sounds that you are looking for, produced by different types of humans (men, women, children, elderly, etc.) all by your own microphone and with high quality. You should also gather a database of the opposite class: those which you wish to omit.
2. Start analyzing the sounds. This might be a little technical and you will need to search for features that can make a reasonable difference between the 2 classes. This might be a combination of features like pitch period, cepstral coefficients, decay rate of harmonic energy,etc. By the way, finding useful features is probably the paramount step of the project.
3. Use a learning scheme ( a classification routine ) like SVM to separate the classes. Actually, you are going to train the machine on your collected database to classify new test inputs.
4. Check out for real, new data. Test your classification with audio frames that are not included in your training database and observe the results. An small percentage of error is usual but large errors should be modified by either modifying the features and training set or changing the classification method.

I hope that helps, and good luck!