My data set contains a number of high resolution greyscale images containing a person's face. A single image containing the background without the face is provided. The background is mostly static in the image, except for some minor changes in lightning from frame to frame. The background pixel intensities can be similar to those of the face. I need an algorithm that would produce a pixel-perfect mask of the face, given that we know what the background looks beforehand.

note:the face outline will not necessarily be as prominent as in this example

(the face outline will not necessarily be as prominent as in this example - edge detection does not find it correctly all the time)

A naive approach I've tried is: run a median filter for noise removal, do a diff of image-background, threshold on low values to produce a binary mask, find biggest contour and fill it in.

However, this fails in cases where:

  1. The background is skin-coloured as image subtraction gives near-0 difference in the face region
  2. The background contains a skin-coloured object right beside the side of the face
  3. The image has a very low contrast. The lower the contrast, the more sensitive it becomes to threshold values, requiring fine tuning for every image.

How would you approach this problem?

  • $\begingroup$ Are the face and the background already co-registered? In other words, are the positions of the camera and background fixed with respect to each other? $\endgroup$
    – A_A
    Sep 28, 2018 at 11:33
  • $\begingroup$ Yes, they are co-registered. $\endgroup$
    – Eustace
    Sep 28, 2018 at 11:47

1 Answer 1


You can try working off a different representation of the image, other than color. You can diff the image gradient for example. It is very unlikely that the gradient is the same on the face as the background. A gradient based descriptor such as HOG or SIFT captured at a suitable scale over the image has worked well for me in the past. You can even combine different representations of the image, such as RGB value as well as HOG. IF the image is different in either mark as a change. You will still get the odd few pixels that don't come up as a change but you can smooth over the result with a smoothing kernel with a similar scale tot he size of the object you want to detect.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.