basic hsb skin detection, neon illumination

Question

i hope this is the right place to ask. otherways sorry for my mistake and pleace advice me a better site.

i'm trying to implement a super simple skin detector using some range of hsb image. i'm using approach described here and here.

i'm try to use a video source from my webcam. if i use sun illumination it works quite well (not so good but quite good), but with neon light.. it is a mess. a lot of white region are detected and a lot of noise everywhere.

why?

i'm using the algorithm described in the second source:

1. convert image ho HSV color space
2. put white on the range 0 < H < 38
3. dilate filter
4. erode filter
5. blur filter

Answer 1

This actually might work better using a simple generative model in RGB rather than HSV.

1. Get a training image or several training images with some skin.
2. Manually select the skin pixels (e. g. by creating a binary mask)
3. Compute the mean and covariance of the skin tone in RGB (each should be 3-element vectors)
4. For an unknown pixel, compute its Mahalanobis distance from the mean, using the covariance.
5. Classify it as skin if the distance is less than a threshold.
6. Tune the threshold for best performance.

Edit: I don't know if OpenCV has a function to compute covariance, but I can tell you how to do it yourself. Let's say you have $n$ RGB pixels. You put them into an $n$ x 3 matrix, let's call it $P$. Then compute $m$, which is the mean RGB vector by calculating the average of the columns. $m$ will be a 1 x 3 vector. Subtract $m$ from every row of $P$ and call the resulting matrix $Q$. Now to compute covariance, all you have to do is multiply $Q$ by the transpose of itself: $C = Q'Q$. Make sure that $C$ is 3 x 3.

Edit2: The values you are getting seem to be too large. To get the maximum covariance create the following matrix:

255 255 255
 0   0   0

and compute covariance of that. You should get a matrix where every value is approximately 32513. So make sure that your pixel values range from 0 to 255, and make sure you copy them into floats or doubles correctly. The Mahalanobis distance is in the units of variance, so for that the numbers should be small. Your threshold for skin classification should probably be less than 4.

Answer 2

Take into account the different values obtained in HSV color when neon light is applied: an example of its deviation is here. Try adapting your algorithm so to it adapt to these values.

Here there is another algorithm to detect skin, and for detecting light conditions you can use this one.

Another algorithm, related to skin detection, but not too much related to neon light effects, is this one.

Answer 3

The answers you've gotten so far point to good alternative methods, but if you're interested in using something like your initial algorithm, it's probably not hard to fix. You just need to adjust for OpenCV's HSV peculiarities. Given the wacky results, I assume you probably used one of the more common numeric representations of HSV in choosing your thresholds and/or in converting pixels?

OpenCV represents HSV differently than most other sources you might have found:

• The biggest difference for you would be w/r/t hue: OpenCV represents hue as ranging from 0 to 179, when almost everything else takes advantage of the hanging bit to preserve more information, w/ 0-255.
• The other difference: measurement of saturation is inverted compared to the norm. So 255 saturation means bright in opencv, instead of white (notice, we're back to 255 - only the hue is 0-180, perhaps because of the 'wheel' representation?)

Probably too late to help you, but it was an interesting question, and someone else might run into the same issue.

Answer 4

import sys
import numpy
import cv2

cap = cv2.VideoCapture(0)
while(1):
    _, im = cap.read()

    im_ycrcb = cv2.cvtColor(im, cv2.COLOR_BGR2YCR_CB)

    skin_ycrcb_mint = numpy.array((0, 133, 77))
    skin_ycrcb_maxt = numpy.array((255, 173, 127))
    skin_ycrcb = cv2.inRange(im_ycrcb, skin_ycrcb_mint, skin_ycrcb_maxt)

    cv2.imshow("Second Image", skin_ycrcb) # Second image
    contours, _ = cv2.findContours(skin_ycrcb, cv2.RETR_EXTERNAL, 
        cv2.CHAIN_APPROX_SIMPLE)
    for i, c in enumerate(contours):
        area = cv2.contourArea(c)
            if area > 1000:
                cv2.drawContours(im, contours, i, (255, 0, 0), 3)
    cv2.imshow("Final Image", im)         # Final image
    cv2.waitKey(1)