Image processing on a surveillance image

Question

I was given this question on a job interview and think I really messed up. I was wondering how others would go about it so I could learn from this experience.

You have one image from a surveillance video located at an airport which includes line of people waiting for check-in. You have to assess if the line is big/crowded and therefore additional clerks are necessary. You can assume anything that may help your answer. What would you do?

I told them I would try to:

1. Segment the area containing people from the rest by edge detection
2. Use assumptions on body contour such as relative height/width to denoise unwanted edges
3. Use color knowledge; but then they asked how to do that and I didn't know

Answer 1

Thanks for sharing your experience. It's a bit off-topic, but my my first advice for a job interview would be to check the company's technology and products. This usually helps a lot in finding answers.

In this general case of crowd surveillance in a closed place, I would check the following items first:

• get a reference image of the empty location in order to be able to detect people's location (even inaccurate) by background subtraction. This gives a rough estimate of the queue length;
• check if the camera is facing the line, in which case I would run a face detector to get an approximate count of people in line. This provides a second estimate;
• if necessary, learn (from the detected faces) the skin color in the image, then again segment the image based on skin color.

There are more advanced techniques available, but I think it would be overkill here.