# Count heart beat from audio capture using smartphone

I am working on a project where I am required to perform heart rate BPM calculation from an audio signal captured using smartphone microphone / hands-free. Constraint is not to use ML. So I am left to play around with filters and shannon.

I am having difficulty cleaning the noise from signal.

Here is what I have so far.

[read audiofile] -> [decimate signal by factor 5] -> [low-pass filter butterworth 350 Hz Order 6] -> [normalize] -> [signal^2 (convolution in TD)]


Use squared signal to workout a threshold and attenuate anything below threshold from the original signal, use shannon energy envelop.

While this pipeline works for most cases when the environment is relatively quiet. But it fails when there is grounding noise or other sharp noises.

I am attaching the samples for anyone who would like to assist. Audio Samples https://app.box.com/s/jli6dut9v4ttrqnmuzjqhkxy701izh2q

I would like help on a) removing grounding noise b) removing crackles c) enhancing beats or removing noise floor in general.

• Good news: even if you were allowed to use ML, basic signal processing is still required and you'd be playing around with: filters and shannon limits :) Nov 21, 2019 at 8:13
• Why and how do you decimate first, from which sample rate to begin with? Why the butterworth filter here? You're not aiming for flat passband, far as I can tell, and a sixth order Butterworth is not a great filter by digital standards. Shouldn't that filter also be decimating? What do you mean with signal^2 (convolution in time domain)? Nov 21, 2019 at 8:16
• Decimating the signal to save on processing power. This eventually will go to a hand-held device. I can do chebychev-I it has a sharper response but I am not sure if a flat band or not so flat band will make any difference here. By signal^2 i meant, multiplying the signal in freq domain by itself. the peaks become more prominent. I am open to suggestions Nov 21, 2019 at 9:35
• the source is always 44kHz In so decimating it by factor 5. Nov 21, 2019 at 9:36
• Why chebychev-I? I mean, that's a reasonable choice for suppression of sidelobe energy, but really, you're nowhere bound to use filters that you could also implement in analog form – simply use your favourite toolbox and design a filter that has the desired cut-off frequency and an appropriate transition width between stop and pass band and an appropriate stopband suppression. You're right to decimate - but you can decimate by a factor of 100 or more instead of 5, and then you'll realize that this uses nearly no processing power at all, and all but the weakest microcontrollers could do this. Nov 21, 2019 at 10:01

This is going to be a long answer.

I'm going to start with an analysis of your files and what you can do to improve the audio. I'll also try your files out on an algorithm I designed to detect heartbeats. After that, I'll describe what went into designing my algorithm and how it works. Then I'll make some general suggestions for how to get started with your own algorithm.

## Analysis of existing files

I loaded each of your files into Audacity to listen to them and to see what the frequency content looks like. In general, you don't merely have a problem with static (white) noise. You have a lot of trouble with constant interference from what looks like the AC power frequency in your area (50Hz and harmonics.)

From worst to best:

File: static_with_beats_doesnt_evaluate.wav

Heavy interference from powerline noise. There is a heartbeat signal in there that I can see. It is too covered up with interference to be detected, though. Nothing much will save this recording. The 50Hz interference is too close to the actual signal to be separated.

File: in_accurate_with_noise.png Here again your biggest problem is interference from the 50Hz power line. At points, the signal is clean enough to pick out a heartbeat. If I run it through a very sharp low pass filter at 45Hz, the algorithm I use can pick out individual beats but even then there are stretches where the heartbeat is simply inaudible.

File: accurate.wav

This file doesn't have the power line interference, probably because it was recorded outdoors. It can be deciphered much better than the others, although there are still sections where there is simply no signal to be detected. A steep low pass filter with a cutoff at 45 Hz gets almost perfect results from my algorithm. There are short sections where the signal is too weak to pick out of the remaining noise.

File: accurate_72bpm.wav

This file is also missing the power line interference. Occasional disturbances cause a beat to be lost here and there, but otherwise a very good recording.

## Problems involved in detecting heartbeats

There are several basic problems involved in detecting heartbeats:

1. What is a heartbeat? A heartbeat is composed of several sounds, most of which are hard to separate from background noises and interference. You have to pick a "feature" than can be reliably picked out of the garbage. Simply detecting the volume of the sound won't do (too many other things can be loud, too.) What I've found is that the "lub dub" sound is a short burst of frequencies between about 30Hz to 40Hz. That's what my algorithm looks for.
2. Threshold - When is have you really detected your feature that says there's been a heartbeat? The volume changes continuously, and so does the volume of the noise and interference. As fixed threshold won't do. Your threshold has to adapt to the background and the changed volume of the heartbeats. The volume of the heartbeats changes with each breath, and it changes as your microphone moves around on the chest. I have a really squirrely solution for the threshold problem.
3. Noise - The algorithm must work in fairly wide range of noisy conditions. The feature you look for should be fairly simple to pick out of nearly any noise. The algorithm I use does well up to about 20dB signal to noise ratio. It is insensitive to the absolute level of the heartbeat signal, but requires a fairly clean signal.
4. Hardware - A functioning algorithm does you no good if it depends on the sensitivity and frequency response of the microphone. Whatever you do needs to work with any microphone on any phone. The algorithm I use is very nearly completely insensitive to the hardware. If the heartbeat is there, it will be found.

Point 3 is really the most important. Loud doesn't matter, absolute level doesn't matter. The whole thing lives and dies on how clean your signal is. Regardless of what algorithm you use, a clean signal is a must.

1. No interference. If you are picking up power line hum, fix it. It's usually an electrical problem and can be solved with proper wiring and grounding.
2. Reduce motion noise. In my experiments, I use an elastic band to hold the microphone on my chest.
3. Reduce background noise. I use a small cardboard "biscuit" stuffed with cotton between the microphone and the elastic band. It is made of several rings of cardboard with a plastic covering. The back is also cardboard. The side facing my chest has only a layer of plastic between the cotton balls and the microphone. That greatly reduces the background noise.

## How I detect heartbeats

The method I eventually came up with is built around some observations of some (really) clean recordings of heartbeats. What I found is that there's a sort of "burst" of frequencies for each "thump" of the heartbeat. If you filter out everything but the content of the burst, then you get a fairly reliable heartbeat detector. I use a bandpass filter for 24 to 44 Hz. That seems to deliver the best results. The range might depend on the person - I don't have enough samples to compare.

After the bandpass, I use a Hilbert transformation to get a continuous estimate of how loud the burst frequencies are. The Hilbert function gives two outputs that are rotated 90 degrees from each other. Square each, take the square root of the sum, and you have a continuous signal representing the amplitude of the "burst" frequencies. Apply a low pass filter to that to get rid of wild variations.

Getting the correct threshold was the real problem, and I tried several ways. I started with manually adjusting the threshold, but of course it changes as you breath and move so that didn't work well. I tried an automatic calibration, which helped but had to be repeated anytime the microphone moved. I tried all kinds of adaptive thresholds, and they all failed for various reasons.

What finally worked really well looks just absolutely stupid.

Take the log of the absolute value of the amplitude signal. Yes, take the logarithm of the amplitude (it doesn't matter what base logarithm you use.) Pass that through a high pass filter, and apply a fixed threshold to the output. That fixed threshold is not setting the absolute level of the signal. It is setting the signal to noise ratio. A lower value means "work with a lower signal to noise ratio" and a higher value means "require a higher signal to noise ratio." The end effect is "trigger if there is a frequency between 24Hz and 44Hz present and it is some fixed ratio louder than the current noise level."

This setup is pretty much insensitive to the absolute levels of the input signal. The frequency response of the microphone and the amplifiers pretty much doesn't matter. It adapts automatically for movement when breathing and when the microphone changes position.

This diagram shows the central part of the process:

That's a snippet of a Pure Data patch that I put together to develop the algorithm. The whole thing together with some example files is available on GitHub. If you want to experiment with it, you will need Pure Data Extended 0.43 to use the patch - it depends on several external libraries that PD Extended includes but which other versions don't.

## Summary

Regardless of how you choose to implement your algorithm, you have to start by looking at the available signal in your recordings.

Look at them in the time domain (oscilloscope view) and the frequency domain (FFT.) Find out what your noise and your interference looks like.

Look at clean recordings and find characteristics that are different from the noise and interference. Find something that can be picked out with a filter of some kind.

Design a filter that can separate the characteristic from the noise.

Devise a method for adapting the threshold to the current noise level and signal level.

My method has drawbacks, of course. The biggest being that if there's no heartbeat it will "imagine" it is hearing one and start delivering false pulses. The independence from the absolute level and hardware variations comes at the cost of having to guarantee a certain minimum signal to noise ratio.