# Kalman filter for tracking obstacles

I'm developing a Computer Vision project in Matlab as an aid to visually impaired people. The setup would be a stereo pair of cameras which the blind person would carry. Using this stereo information, I generate a disparity image using Semi-Global Block Matching (SGBM). After that, I process a "virtual disparity", which basically is a homogeneous transformation, to pass the image plane to the ground, resulting in an image like the following:

Now, After removing the ground plane, morphologically improve the image, and threshold it to keep only near obstacles, I get a foreground mask with the approximate shape of the objects.

I was thinking on tracking each one of the obstacles to improve the detection. Considering that the camera is not fixed since the blind person carrying it is moving, I wonder if the relative movement of all the obstacles in the image would be easily followed by a Kalman filter, or it would be better a feature-based tracker like Kanade–Lucas–Tomasi feature tracker (KLT).

Thank you in advance

## 2 Answers

You might consider a particle filter. Here's a link to a paper I wrote about tracking objects in video using a particle filter. The great thing about these is that objects can be tracked through temporary occlusions. The trick with using a Kalman filter here is dealing with the nonlinearity introduced by the edges of the video field, and casting your measurement model as a linear combination of your states (presumably x,y location). The Unscented Kalman filter is another route you could go, it doesn't require linear models.

Interesting application. Since you have the luxury of depth information, you could quantise that to a small number of levels (8, for instance) and then apply blob detection to recover areas of the image that could qualify as objects. If these obstacles tend to conform to homegeneous or regular shapes (e.g. rectangles, ellipsoids, etc) then you could apply some elementary pattern recognition on each depth zone to recover the locations of potential obstacles.

A Kalman filter could be used to improve tracking in a moving environment but due to paralax it will be difficult to keep tracking various objects when these go out of view (e.g. go behind other objects in the same scene, once you lose the "lock" it will be difficult to re-identify the object as one that was tracked previously and continue tracking it as one target.)

• Thanks, excelent clarification. But when the objects change position, that means they will change also appearance and then pattern recognition will fail, or am I wrong? – agregorio Oct 17 '16 at 12:21
• Thank you. in the case of blob detection, you would be detecting "areas of interest" via some features. For example, roughly rectangular blobs at least $k$ units tall and $m$ units wide. If you were to try to detect objects via their shape, then yes, that would fail if their appearance changed. Happy to ammend the response if more data becomes available. – A_A Oct 17 '16 at 14:13