Removing low frequencies from a signal

Question

I am trying to remove low frequencies from a signal and intuitively I chose the high-pass filter, more specifically - a Butterworth filter, Order 4 (because I am not sure how to choose properly the order and 4 seemed as a good choice) and cutoff frequency of 50 Hz. The problem is, that the filter removes the low frequencies, however, the peaks that were caused from those low frequencies are still to be seen.

I have read also about adaptive filtering, but according to the literature, I need a reference signal, which would be used as a desired signal. I tried implementing an adaptive RLS/LMS filter and as a reference signal, I processed my data with a low-pass filter - Butterworth, 4th Order, 50 Hz for the cutoff frequency. This method also did not get me far.

I have provided a a copy of my data, which is sampled at 1500 Hz.

Answer 1

You want to remove the heart beat signal and keep the "noise". We can solve this problem by using a denoising algorithm, and subtracting the denoised signal from the original signal.

Setting frequency cutoffs for a frequency domain filter can get tricky and turn into a game of whack-a-mole because there's "high frequency" components in the heartbeat blip (due to the sudden rise and fall) and also in the wiggly stuff between a heart beat's T wave and the next heart beat's P wave.

Loosely speaking, the requirements in this de-noising problem are as follows:

• Remove the little wiggles
• Maintain larger jumps that appear in a heart beat PQRST waveform

This sounds like a great place to apply $\ell_1$ denoising or total variation denoising. The idea is to approximate the given signal $y$ with a signal $x$ such that the derivative of $x$ is "sparse" i.e. it doesn't change too frequently, but when it changes, the change is large. The denoised estimate $\hat x$ is given by: $$ \hat x = \min_x ||y-x||_2^2 +\lambda\sum|x_i-x_{i-1}| $$

I used proxTV toolbox in Python to solve this optimization problem.

import scipy.io as sio
import numpy as np
import matplotlib.pyplot as plt
import prox_tv as ptv

mat_struct = sio.loadmat('Signal1.mat')
noisy_signal = mat_struct['x'].T[0]

filtered_signal = ptv.tv1_1d(noisy_signal, 50)

time_vec = np.linspace(0, len(noisy_signal)/1500., len(noisy_signal))

plt.close('all')

fig, ax = plt.subplots(3,1,sharex=True)

ax[0].plot(time_vec,noisy_signal)
ax[0].set_title('noisy signal')

ax[1].plot(time_vec,filtered_signal)
ax[1].set_title('filtered signal')

ax[2].plot(time_vec,noisy_signal - filtered_signal)
ax[2].set_title('noise')
ax[2].set_xlabel('time (s)')

plt.tight_layout()
plt.show(block=False)

And here's the resulting plot:

Of course, there's a different kind of whack-a-mole you'll have to play with this technique: the $\lambda$ parameter which I set to $50$ in the code.