I want to know about Kalman Filter but i tried searching different links including Electrical Engineering StackExchange but the information available there was hardly digestible.

All I am able to understand is that Kalman Filter is an estimator / predictor of future state of any system ,based on a model that is formed by previous measurements / readings.

So please kindly explain in simple words, what is Kalman Filter? What are its applications in Signal / Image processing and how is it different from other filters like Low pass, high pass etc?

  • $\begingroup$ Consider a linear system whose state evolves stochastically due to additive white Gaussian noise (AWGN). Suppose this state is unobservable. What is observable is a noisy linear measurement of the state, also corrupted by AWGN. The Kalman filter finds an estimate of the state that is optimal in some sense. If the measurements are too noisy, they are weighted less, for example. If the system is not linear, one uses variants of the KF. If noise is not AWGN, then one must be careful. The KF is used, for instance, to do "sensor fusion" of GPS and inertial measurements in aircraft and bombs. $\endgroup$ Oct 25 '19 at 8:10
  • $\begingroup$ "all the information I found was hardly digestible": so, what did you understand? What background do you bring? What did you not understand? As you ask this now, you're just asking us to write a text book on signals and systems, and then on advanced control techniques, and put it in an answer here, and that's a bit too broad. $\endgroup$ Oct 25 '19 at 9:00

KF is actually a mixture of a deterministic state propagator and a statistical estimator.

Despite it's name including the term filter, Kalman filter is not a simple frequency selective one. It's indeed a statistical recursive estimator of a state of a (linear) dynamic system. Yet on a broader sense it's called as a filter as it will separate a desired signal (the state) from corrupted measurements; hence filtering.

A linear dynamical system (LDS) is a physical construct whose behaviour is described by (linear) differential equations;

$$ \dot{X} = A X + Bu + W $$

where the vector $X$ is the state of the system, the matrix $A$ is the state propagation information (derived based on the differential equation of the LDS) , $u$ is a vector of deterministic inputs and $W$ is the state noise; depicting the uncertainity in the LDS propagation model $A$.

KF takes advantage of this deterministic behaviour of a LDS before estimating the current state $X$ purely from noisy measurements.

$$ Y = C X + Du + V $$

where $Y$ is the observation about the system and $V$ is the measurement noise on it.

Given a previous estimate of the state, it first deterministically propagates this previous estimate into current estimate through the matrix $A$ (process equation), and then performs a new finer estimate on the difference of the current measurement and propagated estimate using the measurment equation.

  • $\begingroup$ I really like this answer – hence the upvote – but I doubt it's what OP was looking for; if information avaialbale there was hardly digestible applies to all of google, then I think they're just missing the math basics to understand what's happening in a differential equation. $\endgroup$ Oct 25 '19 at 9:34

Perhaps an analogy might be constructive.

Consider a submarine commander with a fat tanker in the cross hairs of his periscope.

He needs to shoot his torpedoes, not at where the target is now, but at some place where the torpedo will intersect with the target.

A skilled commander will have knowledge about how fast or slow the tanker can go. knowledge about how it turns, ect. This is the model.

Looking through the periscope, the commander tries to determine the position, speed, and heading of the target. These are the states of the model. A measurement of the state of the target is less than ideal through a periscope. The commander must make an educated guess of the state.

The commander “predicts” where the target will be and shoots his torpedo. After the expected run time, he raises the periscope and assess the scene. The torpedo ran ahead of the target and missed. The commander decides he had the speed too fast, and the heading was further away. He mentally updates his estimate of the state of the target.

The process repeats with a new attack.

A Kalman Filter doesn’t shoot torpedoes but does use the difference between what the state of the model predicts, and uses the error between the prediction and the measurement to update the estimate of the state.

Ideally there is a “true” model and it’s state is estimated in these recursive “predict-measure-update” cycles.

Typically, models are first order and can produce useful predictions over small steps.

  • $\begingroup$ up arrow from me $\endgroup$ Oct 26 '19 at 1:05
  • $\begingroup$ me too! I think he met the "simple words" metric as requested by the OP $\endgroup$ Oct 27 '19 at 1:58

Simple Description

Imagine you're in a car that is traveling at 70MPH with cruise control. Because the cruise control isn't perfect, your actual speed might vary slightly. This imperfection is called "process noise".

Now lets also imagine the car is being tracked using GPS. Because GPS isn't perfect, there will be some noise in the sensor reading. This imperfection is called "measurement noise".

If we designed a Kalman filter to estimate the actual vehicle position, it would do the following:

  1. Use the process information to predict the next vehicle position.
  2. Take a vehicle position measurement reading.
  3. Calculate the difference between the measurement and the predicted position. This is called the residual.
  4. Takes this residual and weighs it against the process noise and measurement noise to optimally update the position estimate.
  5. Repeat.

So essentially the Kalman filter looks at the discrepancy between our model prediction and measurement and then generates an estimate that leans towards one or the other depending on the process noise and measurement noise.

If we have high process noise (i.e. our cruise control is really bad) and low measurement noise, the Kalman filter will favor the measurement reading when updating the estimate.

If we have low process noise and high measurement noise (very inaccurate GPS reading), the Kalman filter will favor the predicted position when updating the estimate.

The overall idea is that combining information from our process model and measurement will yield a more accurate estimate than using simply one or the other.

Comments on math

The Kalman filter is essentially just an application of recursive least squares where the estimation error covariance is minimized. I would suggest reading up on "least squares estimation" and "recursive least squares estimation".

Also, Kalman "filter" is somewhat of a misnomer. It should be viewed more as an "estimator" that uses statistics to estimate a value. I've always reserved the term "filter" for something that explicitly attempts to influence the frequency content of a signal. While a Kalman filter does influence the frequency content of a signal, I don't believe it does so in an explicit fashion like a high-pass or low-pass filter.


See: What is the relationship between a Kalman filter and polynomial regression?

In over-simplified form, eyeball a line though a cloud of data samples, look where that line might point one sample into the future; and, when you get a new sample check, how good that estimate might have been; then redo, but optimize for a lot less arithmetic per step.

A Kalman filter is usually just an optimization of one, or a set of simple fixed or weighted estimators, such as linear or low order regression fits, plus optional extrapolation and statistical distribution estimation.

So, basically, a statistical summarization of a bunch of previous data into a vastly smaller amount of state variables.

When getting a new ime sample, instead of re-running an entire regression on an entire previous data set plus the one new sample (potentially zillions of inputs and zillions of arithmetic operations), a Kalman filter simply uses the state information left by a previous time step computation plus the one new sample. Using one data input, plus a few state inputs, results in vastly fewer math operations to provide an updated fit and/or statistical estimation, compared to re-running entire regressions on long data sets. Thus also requiring a lot less computation, data storage (computer memory), and bandwidth.

  • $\begingroup$ What once required a Kalman filter on an ancient 1 MIP VAX to keep up with real-time input, can now be done by brute force (recomputing entire regressions every frame) on a typical mobile phone CPU, thus possibly allowing dynamically optimizing a non-linear estimator. $\endgroup$
    – hotpaw2
    Oct 25 '19 at 15:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.