I'm working on a frequency tracking problem with noise, where the amplitude of the noise is orders of magnitude higher than the amplitude of my signal (~1000x). Some details:
- The signal is roughly sinusoidal, less than 1 Hz with non-stationary frequency and amplitude
- The noise is caused by motion, and is (in many cases) not periodic
- I have a 3-channel accelerometer reference for the noise. My target signal has some correlation with motion, so adaptive filtering may have issues
Empirical Mode Decomposition and even simple windowed FFT work well in the no-motion case, but fall apart as soon as motion kicks in - the broadband effects of the noise mask the frequency-domain peak of the signal. I tried RLS adaptive filtering, but I believe the aperiodicity of the noise prevents the filter from converging to anything useful. Would something like Extended Kalman Filtering be able to handle this case? What other options are there when your noise overpowers your signal?
Update
Phase-locked Loops:
According to this article, PLLs...
... can demodulate FM signals with very high accuracy and reliability, or it can detect signals buried in noise, but it can't do both in a single configuration, because the two tasks require very different setups and assumptions. In its role as an FM detector, a PLL doesn't reject noise very efficiently, and as a weak-signal detector, it can't decode FM very efficiently. The reason should be obvious — to detect FM modulation of a given bandwidth, the PLL's feedback loop low-pass filter (Figure 2, green) must be opened up enough to allow the modulation's bandwidth to pass unimpeded, but this causes the PLL to become more susceptible to noise.
Kalman Filtering:
I've been testing out an Unscented Kalman Filter (UKF) for the problem. I found examples of people using Extended Kalman filters, which work ok for the simplest problem of tracking a sine wave because it's easy to derive the Jacobian of a sinusoid. I opted to try the UKF because of the ease of incorporating further non-linear constraints / future problems where the Jacobian isn't easily accessible.
I set up the filter using a state variable of $\begin{bmatrix} A & \omega & \phi & b\end{bmatrix}^T$, where the measured signal associated with the state is $z = A \sin (\omega - \phi) + b$.
The process model assumes that $A$, $\omega$ and $b$ remain constant, and the phase advances according to the current frequency: $\phi[t+1] = \phi[t] + \omega[t] \times dt$ ($\phi$ is constrained to $[-\pi, \pi)$).
The problem I find with this model is that solutions are not unique. For instance, in the case pictured below, the filter has converged by sending the frequency to 0, and manipulating the phase variable to influence the signal:
In each subfigure, the black line represents the estimate from the filter. The top subfigure is the fit of the model against noisy measurements (blue dots). The remaining subfigures show each variable of the state vector - colored dots are the true value that we would like the filter to converge to.
Even if I could restrict the argument of the sin to one process variable, it would always be possible to set the frequency to 0 and use the amplitude or baseline shift to fit the signal.
My sense is there are clever ways of designing the process model to constrain the behavior of the filter. Can anyone suggest a better way to set up the problem?
For reference, I have attached a python 3 notebook with the code I am using.
