Segmenting dark spots from a light background?

Question

I need to perform automatic segmentation of dark blue spots on light yellow paper. Here's a very simple example:

In this case, a simple threshold based on hue or brightness works well. But here are some more challenging real-world examples:

Clearly a simple hue/brightness threshold will fail when dealing with shadows, exposure issues, and small stains where the color is blurred from the background.

I've tried Otsu's method for adaptive thresholding, but it also fails to account for shadows - and produces erratic results when analyzing "blank" cards.

Is there a form of localized thresholding that might work better? I've also considered edge detection, but am unsure how to transform the results into the needed binary image. Any suggestions are appreciated.

Answer 1

It's always hard to give a definitive answer in cases like this, because -- what comes next? We've got 500 million years of evolution behind our visual systems, so sometimes it's hard to realize just how hard of problems we're solving at a glance.

So, if possible, control your source images by not snapping the picture if there's a shadow or stains or whatever!

I'd start by reducing the image dimensions from unsigned RGB to signed B - Y. I.e., for each pixel expressed as $\mathbf x = \begin{bmatrix}r & g & b\end{bmatrix}$, find $x_{by} = \mathbf x \cdot \begin{bmatrix}-\frac 1 2 & -\frac 1 2 & 1\end{bmatrix}$. Double check this visually, but except for the case where the whole scene is washed with blue, your dots should be altogether or mostly positive, and your background should be altogether negative. Of the three colored pictures you show, I think the plain one and the shadowed one should be segmented just by this.

Edge finding on this monochrome image might work -- edge find, then dilate the lines to make sure they're connected, then erode them so they're one pixel wide, then fill in the centers. These should all be in your image processing tool box.

Another thing that might work would be a regional threshold consisting of:

• Filter with a Gaussian filter whose spot size is about as big as your smallest expected blue spot
• Choose a patch size that covers an expected dozen or so spots. Segment the image by thresholding the center of that patch with the minimum and maximum filtered pixel value.
• "Scan" your patch over the whole image, segmenting as you go.

This may still have problems with your sharp-edged shadows, particularly if the shadows are colored. If so, see my comment about controlling the image acquisition process...

To deal with the problem of a blank card coming through the process as having spots -- you want to assess the card for the level of contrast in the image, particularly along that $x_{by}$ axis. I'd compare the overall image brightness with the distance along the $x_{by}$ axis -- if the $x_{by}$ signal is small compared to the image brightness, then this indicates that you're looking at a blank card.