Signal Processing Stack Exchange is a question and answer site for practitioners of the art and science of signal, image and video processing. It only takes a minute to sign up.
Sign up to join this community
I would much appreciate an intuitive explanation for (visual) tracking with Kalman filters. what I know:
Prediction step:
Based on images/measurements $1\rightarrow(t-1)$ I want to predict state $\mathbf x_t$? (using dynamic equation) Is that correct ?
How can I interpret the correction step into those terms (image, target location) ?
First you have to assume a motion model. Let's say you wish to track a ball flying through the air. The ball has a downward acceleration due to gravity of 9.8m/s^2. So in this case the constant acceleration motion model is appropriate.
Under this model, your state is position, velocity, and acceleration. Given the previous state you can easily predict the next state.
You also have a notion of detection. You have a video of the ball moving, and you somehow have to detect the ball in each video frame (e. g. by using background subtraction).
Your detections are noisy. Also, the motion of the ball does not exactly fit the constant acceleration model because of air resistance, wind, cosmic rays, etc. The Kalman filter needs two matrices describing this: one is the covariance of the measurement noise (your detection inaccuracy), and one for the covariance of the process noise (how the motion of the ball deviates from the model you have specified).
If you are tracking a single object, then the Kalman filter lets you smooth out some of the noise, and also predict where the object is when a detection is missing (e. g. if the object is occluded). Here is an example of tracking a single object with a Kalman filter using the Computer Vision System Toolbox for MATLAB.
If you are tracking multiple objects, then the Kalman filter predictions let you decide which detection goes with which object. A good way to do that is to use the log likelihood of the detection given the error covariance of the prediction. Here is an example of tracking multiple objects with a Kalman filter.
This online course is very easy and straightforward to understand and to me it explained Kalman filters really well.
It's called "Programming a Robotic Car", and it talks about three methods of localiczation: Monte Carlo localization, Kalman filters and particle filters. It does focus on sonar information as an example, but the explanation is simple enough that you could simply replace "sonar" with "visual information" and it would all still make sense.
The course is completely free (it's finished now so you can't actively participate but you can still watch the lectures I presume), taught by a Stanford professor.
The Kalman filter recursively provides the optimal linear estimate of a signal perturbed by AWGN. In your case, the state (what you want to estimate) will be given by the target location. The measurements will be determined by your algorithm.
If you've read the Wikipedia article, you might like to view this presentation on visual tracking. Do you have any books?
When you are doing visual tracking you need a model, which is a mathematical representation of a real-world process. This model will give sense to any data obtained from measurements, will connect the numbers we put into and we get out of the system.
But a model is a simplification of reality because you will use a reduced number of parameters. What you don't know about the system is called noise or uncertainty. It is as important as what you know. As we cannot describe a system perfectly, we need measurements from the real world to tell us what is happening to the system we are modelling.
Kalman is a tool for combining what we estimate, with our model, and what we measure from the world, by combining both in a weighted sense.
You will calculate a state every step. That's what you currently know about the system. The state is influenced by the process equation and the measurement equation. Both equations have different noise covariances. Kalman will decide which of both has more influence each step by adjusting the Kalman gain.
This is the way I think about this when I don't wan't to get deep into formulas.
