I have a live signal that needs filtering, and I'm trying to compare a Kalman filter with a simple exponential filter. For this I generated an artificial ground truth signal, add some noise and filter it with both methods.

The problem is that, while the result from the Kalman filter 'looks' better, I can't quantify how much better it is because its output lags behind the signal. When I simply compute the error between a noisy sample and a filtered sample at time T, the difference is always higher than between the noisy sample and the ground truth sample at the same time.

Is there a way of correctly computing the error in these cases? I thought about computing a least squares fit between the filtered and ground truth signals, but that feels like 'cheating' a bit, as it looks at the best possible case and also doesn't account for the filter lag.

Edit: Some graphs to give a better picture of what I'm doing Ground truth, noisy signal and filter outputs

I've plotted the outputs up to t=140. Both the exponential filter (red) and kalman filter (green) are lagging behind the current noisy sample (blue). If I try to compute the squared error by comparing the ground truth value at t=140 with kalman or exponential values at t=140, the error will be large, and at times even larger than the error between the ground truth and the noisy signal (second picture top).

This makes the exponential filter look better, just because it has less lag than the kalman filter. If I try to account for lag, i.e. compute the error between the kalman filter at t=140 and ground truth at the closest t ~ 137-ish, then the squared error graphs look better (second picture bottom), but I'm not sure if this is the correct thing to do.

square error between the ground truth and noisy/filtered signals

Edit 2 I now realize that these are not really time series, but rather processed as a 2D signal. So 't=' should really be 'x=', with the value being y. This explains why the sequences are not synchronized in time. But I think the question is still valid.

  • $\begingroup$ You should probably make a clear distinction here between filters and smoothers. Filters are causal and smoothers are not. It is unfair to compare a Kalman filter even with the most simple smoother (because it just knows the future!). If you are doing real-time stuff, compare your Kalman filter with a simple low-pass filter, but if not using Kalman filters is probably not the best way to go $\endgroup$ – gsmafra Aug 1 '15 at 17:58
  • $\begingroup$ My exponential smoother just knows the current and previous samples, i.e. y(t)=a*x(t)+(a-1)*y(t-1). I've named it 'smoother' because wikipedia said so :) $\endgroup$ – John S Aug 1 '15 at 22:58
  • $\begingroup$ I call this a 1st order IIR linear fiter. The correct formula should use $(1-a)$ instead of $(a-1)$. All causal filters should lag behind the real signal, the KF should be actually better because it also does prediction, so if you have worse results with it then there is probably a tuning problem. About the criteria, it depends on what you consider a good filter to be but there's no problem IMO in computing the MSE between filtered signal and ground truth and you don't need to account for the lag. As I said, the KF is also a predictor so in the best situation it will actually correct the lag. $\endgroup$ – gsmafra Aug 1 '15 at 23:18
  • $\begingroup$ PS: I don't know what you mean by "looks at the best possible case" and no, it wouldn't be cheating, this is just supervised learning. The problem is that when you put the KF to action the real measurement noise and dynamics of the system will be different from what you've synthesized and therefore the tuning will not be perfect. $\endgroup$ – gsmafra Aug 1 '15 at 23:40
  • $\begingroup$ You are right, the formula should be (1-a). I've computed the error between the filtered signal at time t and the closest sample of the ground truth with regard to t (in the past), the results seem to show the kalman filter is indeed superior, with a low consistent error compared with the exponential filter which is all over the place. I agree that the problem with lag most likely comes from the parameters of the kalman filter. It never overshoots, even if there is an abrupt change in the signal (step). Are there any best practices for tuning the matrix values? ...(continued) $\endgroup$ – John S Aug 2 '15 at 14:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.