Producing masks from mostly-static-background subtraction when given the background

Question

My data set contains a number of high resolution greyscale images containing a person's face. A single image containing the background without the face is provided. The background is mostly static in the image, except for some minor changes in lightning from frame to frame. The background pixel intensities can be similar to those of the face. I need an algorithm that would produce a pixel-perfect mask of the face, given that we know what the background looks beforehand.

(the face outline will not necessarily be as prominent as in this example - edge detection does not find it correctly all the time)

A naive approach I've tried is: run a median filter for noise removal, do a diff of image-background, threshold on low values to produce a binary mask, find biggest contour and fill it in.

However, this fails in cases where:

1. The background is skin-coloured as image subtraction gives near-0 difference in the face region
2. The background contains a skin-coloured object right beside the side of the face
3. The image has a very low contrast. The lower the contrast, the more sensitive it becomes to threshold values, requiring fine tuning for every image.

How would you approach this problem?

Answer 1

You can try working off a different representation of the image, other than color. You can diff the image gradient for example. It is very unlikely that the gradient is the same on the face as the background. A gradient based descriptor such as HOG or SIFT captured at a suitable scale over the image has worked well for me in the past. You can even combine different representations of the image, such as RGB value as well as HOG. IF the image is different in either mark as a change. You will still get the odd few pixels that don't come up as a change but you can smooth over the result with a smoothing kernel with a similar scale tot he size of the object you want to detect.