When Is a Kalman Filter Different from a Moving Average?

Question

this thread asks when a discrete time Kalman filter is better/different from a simple moving average of the observations:

Why use a Kalman filter instead of keeping a running average?

there's no definitive answer. can someone give a definitive example where the kalman filter, ideally in simple 1D case, does something different (and better) than keeping a moving average, and state the conditions when the kalman filter would reduce to a simple moving average?

one thought is that the kalman filter would not weigh all the data points equally because its variance is initially smaller and gets better with time. but it sounds like that would only matter close to initial observations and that once the variance converged, the kalman filter would weigh each observation equally just like a moving average, so don't see when the two are different and why/when the filter would do better.

Answer 1

Another take: The Kalman Filter lets you add more information about how the system you're filtering works. In other words, you can use a signal model to improve the output of the filter.

Sure, a moving average filter can give very good results when you're expecting a close-to-constant output. But as soon as the signal you're modelling is dynamic (think speech or position measurements), then the simple moving average filter will not change quickly enough (or at all) compared with what the Kalman Filter will do.

The Kalman filter uses the signal model, which captures your knowledge of how the signal changes, to improve its output in terms of the variance from "truth".

Answer 2

as the first answer (with the most votes) says, the kalman filter is better in Any case when signal is changing. Notice the problem statement These use the algorithm to estimate some constant voltage. How could using a Kalman filter for this be better than just keeping a running average? Are these examples just oversimplified use cases of the filter? using a kalman filter to estimate a constant voltage is definitely, overkill. In that particular problem it is better to use the running average, which we know is the best estimator for Gaussian distributions. In this example the measured voltage is the actual voltage V but with some noise typically modeled as 0 mean Gaussian (white noise). so that our measurements are Gaussian with mean=V, and sigma=sigma noise.

The kalman filter is better suited for estimating things that change over time. The most tangible example is tracking moving objects. Let's imagine throwing a ball, we know it will make a parabolic arc, but what will our estimators show?

A Kalman filter will be very close to the actual trajectory because it says the most recent measurement is more important than the older ones (when the covariance is low that is). The running average takes all measurements equally

blue-ball trajectory, red-running average

(sorry no kalman if i have time i'll throw it in there if i have time, but it would me much closer to the blue line assuming you modeled the system well)

equations courtesy of wikipedia

The kalman filter on the other hand says, if our convariance and residual were small (meaning we had a good estimate), then we are going to stick with out previous estimate and tweak it a tiny bit based on the residual(or our estimation error). Now since our xhat k|k is very close to the actual state, when we make out next update, we will use a system state that closely matches the actual state.

At x=30, the running average says, the initial condition y(0) is just as important as y(29), thats that, and you get a huge error. The kalman filter accounted for this. It said since our error last time was huge, lets make a drastic change to our estimate (our xhat) so when we use it for the next update, it will be closer to what is actually happening

I hope that makes some sense

-------------------------EDIT--------------------------------

I just noticed your question asks about a moving average vs kalman. I answered running avg vs kalman (that is the topic of the link you provided)

Just to add a little more info specifically to the moving(windowed) average. The moving average is a better estimator of changing values. Since it only takes into account more recent samples. Unfortunately, it has a lag associated with it, especially around changing derivatives (Just look near t=30, where the derivative is going from positive to negative). This is because the average is slow to see fluctuation. Which is typically why we use it, to remove fluctuation (noise). The window size also plays a role. A smaller window is usually closer to the measured values, which makes sense and sounds good, right? The downside of this is if you have noisy measurements, a small window means more noise shows up more in the output. Let's look at the other question again

measurements with mean = .5, sigma = .1;

z = [0.3708435, 0.4985331, 0.4652121,...

the average of the first 3 samples is 0.4448629 not exactly close to the .5 expected value. This again shows, that with the smaller window, noise has a more profound effect on the output.

So then logically our next step is to take larger windows, to improve our noise immunity. Well, turns out larger windows are even slower to reflect actual changes (again look at t=30 in my graph) and the most extreme case of windowing is basically the running average (which we already know is bad for changing data)

Now back to the magical kalman filter. If you think about it it similar to a 2 sample windowed average (similar not the same). Look at X k|k in the update step, it takes the previous value, and adds to it a weighted version of the current sample. You might think, well what about noise? Why isn't it susceptible to the same problem as windowed average with a small sampling size? Because the kalman filter takes into account the uncertainty of each measurement. The weighting value K (kalman gain) can be though of as a ratio between the covariance(uncertainty) of your estimate and the covariance (uncertainty) of the current estimate (actually its the residual, but it's easier to think of it this way). So if the latest measurement has lots of uncertainty K decreases, and thus the most recent sample plays a smaller roll. If the latest measurement has less uncertainty than the prediction, k increases, and now the new information plays a bigger roll in the next estimate. So even with a small sample size, the kalman filter is still blocking out a lot of the noise.

anyway, I hope that answers the "windowed avg vs kalman" question now

Answer 3

I won't add any equations, I will just add some intuition.
I will also limit my self for Additive Gaussian White Noise.

Now, in that case the Kalman filter can written as a Least Squares problem to solve.
I'd say even more, the Kalman Filter is linear, if you have the samples up to certain time $ T $, you can write the Kalman filter as weighted sum of all previous and the current samples. Moreover, if we assume there is no process noise, it collides with the Least Squares regression filter.

So what's make it so special? 3 things are making it special:

• Its Form - as Recursive filter it is easy to implement in limited computing resources.
• The Process Noise Concept - Namely, it doesn't assume the model is known perfectly, this makes it adaptive filter which corrects its weighting each iteration (For all samples!!!).
• Non Stationary Noise - The ability to deal with non stationary process, namely process which its noise (Both the process and measurement) changes over time.