Kalman filter for tracking obstacles

Question

I'm developing a Computer Vision project in Matlab as an aid to visually impaired people. The setup would be a stereo pair of cameras which the blind person would carry. Using this stereo information, I generate a disparity image using Semi-Global Block Matching (SGBM). After that, I process a "virtual disparity", which basically is a homogeneous transformation, to pass the image plane to the ground, resulting in an image like the following:

Now, After removing the ground plane, morphologically improve the image, and threshold it to keep only near obstacles, I get a foreground mask with the approximate shape of the objects.

I was thinking on tracking each one of the obstacles to improve the detection. Considering that the camera is not fixed since the blind person carrying it is moving, I wonder if the relative movement of all the obstacles in the image would be easily followed by a Kalman filter, or it would be better a feature-based tracker like Kanade–Lucas–Tomasi feature tracker (KLT).

Thank you in advance

Answer 1

You might consider a particle filter. Here's a link to a paper I wrote about tracking objects in video using a particle filter. The great thing about these is that objects can be tracked through temporary occlusions. The trick with using a Kalman filter here is dealing with the nonlinearity introduced by the edges of the video field, and casting your measurement model as a linear combination of your states (presumably x,y location). The Unscented Kalman filter is another route you could go, it doesn't require linear models.

Answer 2

Interesting application. Since you have the luxury of depth information, you could quantise that to a small number of levels (8, for instance) and then apply blob detection to recover areas of the image that could qualify as objects. If these obstacles tend to conform to homegeneous or regular shapes (e.g. rectangles, ellipsoids, etc) then you could apply some elementary pattern recognition on each depth zone to recover the locations of potential obstacles.

A Kalman filter could be used to improve tracking in a moving environment but due to paralax it will be difficult to keep tracking various objects when these go out of view (e.g. go behind other objects in the same scene, once you lose the "lock" it will be difficult to re-identify the object as one that was tracked previously and continue tracking it as one target.)