22
$\begingroup$

Background: I'm working on an iPhone application (alluded to in several other posts) that "listens to" snoring/breathing while one is asleep and determines if there are signs of sleep apnea (as a pre-screen for "sleep lab" testing). The application principally employs "spectral difference" to detect snores/breaths, and it works quite well (ca 0.85--0.90 correlation) when tested against sleep lab recordings (which are actually quite noisy).

Problem: Most "bedroom" noise (fans, etc) I can filter out through several techniques, and often reliably detect breathing at S/N levels where the human ear cannot detect it. The problem is voice noise. It's not unusual to have a television or radio running in the background (or to simply have someone talking in the distance), and the rhythm of voice closely matches breathing/snoring. In fact, I ran a recording of the late author/storyteller Bill Holm through the app and it was essentially indistinguishable from snoring in rhythm, level variability, and several other measures. (Though I can say that apparently he didn't have sleep apnea, at least not while awake.)

So this is a bit of a long shot (and probably a stretch of forum rules), but I'm looking for some ideas on how to distinguish voice. We don't need to filter the snores out somehow (thought that would be nice), but rather we just need a way to reject as "too noisy" sound that is overly polluted with voice.

Any ideas?

Files published: I've placed some files on dropbox.com:

The first is a rather random piece of rock (I guess) music, and the second is a recording of the late Bill Holm speaking. Both (which I use as my samples of "noise" be differentiated from snoring) have been mixed with noise to sort of obfuscate the signal. (This makes the task of identifying them significantly more difficult.) The third file is ten minutes of a recording of yours truly where the first third is mostly breathing, middle third is mixed breathing/snoring, and the final third is fairly steady snoring. (You get a cough for a bonus.)

All three files have been renamed from ".wav" to "_wav.dat", since many browsers make it maddeningly difficult to download wav files. Just rename them back to ".wav" after downloading.

Update: I thought entropy was "doing the trick" for me, but it turned out to mostly be peculiarities of the test cases I was using, plus an algorithm that wasn't too well designed. In the general case entropy is doing very little for me.

I subsequently tried a technique where I compute the FFT (using several different window function) of the overall signal magnitude (I tried power, spectral flux, and several other measures) sampled about 8 times a second (taking the stats from the main FFT cycle which is every 1024/8000 seconds). With 1024 samples this covers a time range of about two minutes. I was hoping that I would be able to see patterns in this due to the slow rhythm of snoring/breathing vs voice/music (and that it might also be a better way to address the "variability" issue), but while there are hints of a pattern here and there, there's nothing I can really latch onto.

(Further info: For some cases the FFT of signal magnitude produces a very distinct pattern with a strong peak at about 0.2Hz and stairstep harmonics. But the pattern is not nearly so distinct most of the time, and voice and music can generate less distinct versions of a similar pattern. There might be some way to calculate a correlation value for a figure of merit, but it seems that would require curve fitting to about a 4th order polynomial, and doing that once a second in a phone seems impractical.)

I also attempted to do the same FFT of average amplitude for the 5 individual "bands" I've divided the spectrum into. The bands are 4000-2000, 2000-1000, 1000-500, and 500-0. The pattern for the first 4 bands was generally similar to the overall pattern (though there was no real "stand-out" band, and often vanishingly small signal in the higher frequency bands), but the 500-0 band generally was just random.

Bounty: I'm going to give Nathan the bounty, even though he's not offered anything new, given that his was the most productive suggestion to date. I still have a few points I'd be willing to award to someone else, though, if they came through with some good ideas.

Improve this question
$\endgroup$
15
  • 2
    $\begingroup$ Can you post some representative spectrograms? (Let the data talk.) How are you computing the "spectral difference"? $\endgroup$
    – Emre
    Mar 24, 2012 at 22:20
  • $\begingroup$ The "spectral difference" is sometimes referred to as "spectral flux" and a few other terms. Basically it's the sum of the squares of individual differences in successive FFTs of the sound. $\endgroup$
    – Daniel R Hicks
    Mar 25, 2012 at 0:10
  • 2
    $\begingroup$ Come to think of it, audio clips would be even better. Thanks for the clarification. $\endgroup$
    – Emre
    Mar 25, 2012 at 0:16
  • $\begingroup$ I'm trying to figure out what the best presentation would be. I don't have fancy software to do spectra (other than my actual FFTs) for short samples -- Audacity requires too long of a sample to capture the spectrum of an individual sound. And there are some medical privacy issues with posting actual patient sounds. $\endgroup$
    – Daniel R Hicks
    Mar 25, 2012 at 0:22
  • 1
    $\begingroup$ @DanielRHicks Spectrograms with TV+snore and just snore or somethings like that would really go a long way. $\endgroup$
    – Spacey
    Mar 25, 2012 at 3:37

4 Answers 4

Reset to default
11
$\begingroup$

Background

According to papers below, snoring is characterized by a peak at about 130Hz, and is wholly concentrated below 12kHz:

Let's see if we can leverage this.

MATLAB Example

We have a poor recording of a child snoring; a 10-minute, 8-bit mono WAV file. The sampling rate is 8KHz, which means the bandwidth of the audio signal is 4KHz. The level is very low so I will compand it first.

[snd,fs]=wavread('recordedFile20120408010300_first_ten_minutes');
cmp=compand(snd,255,1);
wavwrite(cmp,'companded'); % used for listening purposes
[s,f,t,p]=spectrogram(snd,hann(8192));
surf(linspace(0,600,length(t)),f/pi,10*log10(p),'edgecolor','none'); 
axis tight; view(0,90);

Full spectrogram

The y axis is normalized to the bandwidth, 4KHz, so that notch you see at 0.1 corresponds to a frequency of 400Hz. There is a spike corresponding to a cough at ~186s; ignore that. We can vaguely see the notches during each snore. Not only that, but they seem concentrated below 0.2 x 4KHz=800Hz. Let's take a closer look.

idx_max_freq=round(0.2*length(f));
surf(linspace(0,600,length(t)),fs*f(1:,idx_max_freq:)/(2*pi),10*log10(p(1:idx_max_freq,:)),'edgecolor','none');
axis tight; view(0,90);

Zoom of the spectrogram to 0-800Hz.

This time the frequency axis was labeled in Hertz. Now the notches are quite clear. We can even see the overtones of the power line noise starting at 60Hz (180Hz, 300Hz, 420Hz). Now comes the essence of the algorithm: let's classify the signal based on the energy in this subband, with the line noise removed.

freq_list=round([1:57 63:177 183:297 303:417 423:800]*idx_max_freq/800);
y=10*log10(sum(p(freq_list,:)));
plot(linspace(0,600,length(y)),y-median(y))
stem(linspace(0,600,length(y)),y-median(y)>.5*std(y))

If we want to get fancy, we can discard oversized spikes:

stem(linspace(0,600,length(y)),(y-median(y)>.5*std(y)).*(y-median(y)<5*std(y)))

Plot of the energy in the notched subband 0-800Hz

The final result

The low SNR, manifest in the difficulty in discerning the signal in the first plot, means we have leeway of only half a standard deviation (the value of which was 4.1). The stems mark the snores.

Improve this answer
$\endgroup$
8
  • $\begingroup$ Yeah, my current algorithm cancels hum and other harmonic noise (ventilator noise tends to be about 110Hz) by zeroing out the FFT buckets that are at a more or less constant level. I'm unclear what you mean when you say "let's classify the signal based on the energy in this subband" -- what subband are you referring to? $\endgroup$
    – Daniel R Hicks
    May 25, 2012 at 0:42
  • $\begingroup$ Oh, I see, you're talking about below 800 Hz -- I missed that bit. $\endgroup$
    – Daniel R Hicks
    May 25, 2012 at 0:43
  • $\begingroup$ If you look at your upper chart you'll see that there's quite a bit of information near the top, and another band just below the middle. And these bands have much less competing noise. My current scheme slices the spectrum and attempts to evaluate the SNR of each slice, then weights them accordingly. $\endgroup$
    – Daniel R Hicks
    May 25, 2012 at 0:48
  • $\begingroup$ You can concatenate several features, such as the energy in select subbands, the spectral flatness, and so on to create a tentative feature vector. Then perform PCA to find out which ones matter most, as explained in the last paper. $\endgroup$
    – Emre
    May 25, 2012 at 0:59
  • $\begingroup$ That's basically what I'm doing, absent the rigor of the PCA. $\endgroup$
    – Daniel R Hicks
    May 25, 2012 at 1:33
9
+100
$\begingroup$

Just throwing this in here to cover all the possibilities, you might be able to use entropy, I don't known what the entropy level of snoring vs speech is but if it is different enough that may work. http://www.ee.columbia.edu/~dpwe/papers/ShenHL98-endpoint.pdf

Improve this answer
$\endgroup$
5
  • $\begingroup$ I don't entirely understand what I'm doing with it (information theory makes my head explode), but I implemented a crude entropy calculation, threw in a bit of totally non-theoretical normalization, and it seems to be working. Music and voice have a low (negative) entropy, while snoring is significantly higher. And background noise in general seems to decrease the (negative) value, so it provides a pretty good figure of merit. More experimentation needed, though. $\endgroup$
    – Daniel R Hicks
    Mar 27, 2012 at 18:04
  • $\begingroup$ One thing to be careful of is you also need to include a simple level gate as well since, you can have extremely low level back noise which may match the stuff you are trying to let through, also my experience seems to suggest the digit quantisation of extreme low level noise can increase the about of order in the signal, since at low levels there are a reduced number of possible sample values and entropy alone does not consider differences in amplitude. $\endgroup$
    – Nathan Day
    Mar 27, 2012 at 22:20
  • $\begingroup$ Yeah, I'm regularly doing battle with noise, and have a pretty good way to separately gauge that. When noise exceeds a certain level I punt. (It's amazingly difficult, actually, to measure noise.) $\endgroup$
    – Daniel R Hicks
    Apr 2, 2012 at 18:12
  • $\begingroup$ Alas, I've discovered that much of what I was measuring with my crude entropy calculation was an artifact of the calculation (due to zeros in the test data). It sorta works for my need, but not as nicely as I first thought. $\endgroup$
    – Daniel R Hicks
    Apr 24, 2012 at 1:12
  • $\begingroup$ Follow-up: I replaced the fixed/float FFT I'd been using with a full floating point one (that doesn't produce zeros when levels are low), and the usefulness of entropy went further down the tubes -- doesn't appear to provide anything particularly useful. $\endgroup$
    – Daniel R Hicks
    Jun 8, 2012 at 13:26
3
$\begingroup$

Time domain statistics perhaps? Snoring seem to have relatively long periods of steady state whereas speech energy changes quite a bit over short time periods. This could be combined with spectral analysis as well. Vowels have more low frequency content and consonants more high frequencies. During speech the spectrum may quickly bounce back and forth between those states while storing may stay in one state for longer periods of time.

Improve this answer
$\endgroup$
6
  • $\begingroup$ The basic time-domain stats are indistinguishable. However, it's a good point that I could look at the short-term variability (which I normally smooth out). Looking for the "bouncing" between frequency bands is also a good idea ... I currently divide into 5 bands and reject bands with apparent low S/N. $\endgroup$
    – Daniel R Hicks
    Mar 25, 2012 at 0:14
  • $\begingroup$ @DanielRHicks I saw how to compute cepstral envelope somewhere else, but perhaps you can use that as a measure of your spectral variability instead of the pure spectrum which will be more 'noisy/jaggedy' whereas the cepstrum(s) tend to me more smooth. I have also heard Mel-Frequency cepstrum is used in voice recog, and this sounds like it might me of use to you. $\endgroup$
    – Spacey
    Mar 25, 2012 at 3:36
  • $\begingroup$ @DanielRHicks: Indistinguishable by what method? They're certainly distinguishable to me. $\endgroup$
    – endolith
    Mar 25, 2012 at 19:09
  • $\begingroup$ @endolith -- Indistinguishable given my current metrics -- the "spectral difference" plus overall energy level. But they are low-pass filtered with about a 0.5 second time constant. I think I'll try looking at the unfiltered data a bit. $\endgroup$
    – Daniel R Hicks
    Mar 26, 2012 at 11:50
  • $\begingroup$ I had a go at capturing the short-duration time stats. Some "hints", but nothing definitive. $\endgroup$
    – Daniel R Hicks
    Mar 27, 2012 at 18:12
1
$\begingroup$

Spectral complexity over time. I'll hypothesize that human speech probably uses more phonemes, and with much greater statistical complexity in their sequencing than the phoneme sequences of snoring.

This is an probably a much easier problem than continuous speech recognition, as you won't need to actually recognize any particular phoneme or sentence correctly, only the number of phoneme sounding spectral segments, and some statistical complexity measure of their sequences (an entropy or compressibility test might work). Then see if you can determine a reliable threshold for these measures.

Improve this answer
$\endgroup$
4
  • $\begingroup$ The problem is that snoring is amazingly complex/random, and lacks much in the way of distinct features when its spectrum is examined. $\endgroup$
    – Daniel R Hicks
    Mar 26, 2012 at 11:53
  • $\begingroup$ It would be interesting information if a sleeping person formed as many different vowel forment filters and plosives (and digraphs and trigraphs of such at a density over time), and pitch inflections (etc.), while sleeping as when awake and talking. $\endgroup$
    – hotpaw2
    Mar 26, 2012 at 13:55
  • $\begingroup$ The lack of features can be a significant feature. Speech has features. $\endgroup$
    – hotpaw2
    Mar 26, 2012 at 14:01
  • $\begingroup$ One problem is that snoring can vary immensely, from one breath to the next. A simple heavy breath is very "white", but a snore may have some very strong spikes. It's essentially a square wave, though that's an over-simplification. And we need to do the analysis in real time on a smart phone, so the complexity of the algorithm is limited. $\endgroup$
    – Daniel R Hicks
    Mar 27, 2012 at 1:43

Not the answer you're looking for? Browse other questions tagged or ask your own question.