How can I track detected objects from frame to frame

Question

I want to track people's faces in a video stream (like a security camera), but I am running into a problem. I was using a Haar classifier to find the faces and sending the detected faces to a system to identify the person that was detected (so I can display their name under there face). The problem is that identifying a person takes more than a second, too slow and unnecessary to do every frame as the same people tend to stick around.

• What is a good way to track people frame to frame? I was thinking of something like storing every person, and in the next frame trying to find faces detected very close to where they were, but this does not work very well as faces can skip frames, move more than I expect, or "jump" faces swapping who is who.

• Is there an accepted way to solve this problem?

• What are some ways to overcome this?

Answer 1

Object detection is relatively a heavy task as you've notice. Detecting the object (in your case human face) in every and each frame would be cumbersome and computationally immense. Therefore, you need to employ an object tracking technique. There are various tracking algorithms, of which, KLT and mean-shift are the two popular ones. KLT works based on object features (corner,..) but mean-shift (usually) is implemented based on intensity of image. Personally, I recommend mean-shift for your application, because I think face color would be more unique in a image sequence than general features (corner,edge,..) of the face. The only thing you might need to add, is verification of the tracker by your object detection algorithm, every like 50 frames to ensure sound functionality of the tracker.

For implementations see:

http://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_video/py_meanshift/py_meanshift.html

http://www.pyimagesearch.com/2015/09/14/ball-tracking-with-opencv/