28
$\begingroup$

Here is a question for image processing experts.

I am working on a difficult computer vision problem. The task is to count the stomata (marked below) in DIC microscopy images. These images are resistant to most superficial image processing techniques like morphological operations and edge detection. It is also different from other cell counting tasks.

I am using OpenCV. My plan is to review potentially useful features for stomata discrimination.

  • Texture classifiers
    • DCT (Discrete cosine transform/frequency-domain analysis)
    • LBP (Local binary patterns)
  • HOG (Histogram of oriented gradients)
  • Robust feature detectors (I am skeptical)
    • Harris corners
    • SIFT, SURF, STAR, etc.
  • Haar cascade classifier/Viola-Jones features

And possibly design a novel feature descriptor. I am leaving out the selection of a classifier for now.

What have I missed? How would you solve this? Solutions for similar object detection problems would be very helpful.

Sample images here.

stomata

After bandpass filter: bandpass filtered

Canny edge detection is not promising. Some image areas are out of focus: canny edge detection

$\endgroup$
3
  • 1
    $\begingroup$ Maybe instead of trying to find the stomata, you could try to remove the mazy lines? $\endgroup$
    – endolith
    Nov 22, 2011 at 1:26
  • 1
    $\begingroup$ How many images do you have to process? How fast does it need to be? How automated does it have to be? $\endgroup$
    – endolith
    Nov 22, 2011 at 20:36
  • 1
    $\begingroup$ It does not have to be very fast. We are processing on the order of 1000 images. It should be automatic - dump images into a directory and go. $\endgroup$
    – Matt M.
    Nov 23, 2011 at 3:03

4 Answers 4

15
+100
$\begingroup$

Sorry I don't know OpenCV, and this is more a pre-processing step than a complete answer:

First, you don't want an edge detector. An edge detector converts transitions (like this dark-to-light):

    _____/‾‾‾‾‾

into ridges (bright lines on dark) like this:

    ____/_____

It performs a differentiation, in other words.

But in your images, there is a light shining down from one direction, which shows us the relief of the 3D surface. We perceive this as lines and edges, because we're used to seeing things in 3D, but they aren't really, which is why edge detectors aren't working, and template matching won't work easily with rotated images (a perfect match at 0 degrees rotation would actually cancel out completely at 180 degrees, because light and dark would line up with each other).

If the height of one of these mazy lines looks like this from the side:

    ____/_____

then the brightness function when illuminated from one side will look like this:

    ____∧v____

This is what you see in your images. The facing surface becomes brighter and the trailing surface becomes darker. So you don't want to differentiate. You need to integrate the image along the direction of illumination, and it will give you the original height map of the surface (approximately). Then it will be easier to match things, whether through Hough transform or template matching or whatever.

I'm not sure how to automate finding the direction of illumination. If it's the same for all your images, great. Otherwise you'd have to find the biggest contrast line and assume the light is perpendicular to it or something. For my example, I rotated the image manually to what I thought was the right direction, with light coming from the left:

original, rotated

You also need to remove all the low-frequency changes in the image, though, to highlight only the quickly-changing line-like features. To avoid ringing artifacts, I used 2D Gaussian blur and then subtracted that from the original:

high pass filtered

The integration (cumulative sum) can runaway easily, which produces horizontal streaks. I removed these with another Gaussian high-pass, but only in the horizontal direction this time:

output

Now the stomata are white ellipses all the way around, instead of white in some places and black in others.

Original:

enter image description here

Integrated:

enter image description here

from pylab import *
import Image
from scipy.ndimage import gaussian_filter, gaussian_filter1d

filename = 'rotated_sample.jpg'
I = Image.open(filename).convert('L')
I = asarray(I)

# Remove DC offset
I = I - average(I)

close('all')
figure()
imshow(I)
gray()
show()
title('Original')

# Remove slowly-varying features
sigma_2d = 2
I = I - gaussian_filter(I, sigma_2d)

figure()
imshow(I)
title('2D filtered with %s' % sigma_2d)

# Integrate
summed = cumsum(I, 1)

# Remove slowly-changing streaks in horizontal direction
sigma_1d = 5
output = summed - gaussian_filter1d(summed, sigma_1d, axis=1)

figure()
imshow(output)
title('1D filtered with %s' % sigma_1d)

The Hough transform can be used to detect ridge ellipses like this, made of "edge pixels", though it's really expensive in computation and memory, and they are not perfect ellipses so it would have to be a bit of a "sloppy" detector. I've never done it, but there are a lot of Google results for "hough ellipse detection". I'd say if you detect one ellipse inside the other, within a certain size search space, it should be counted as a stoma.

Also see:

$\endgroup$
2
  • $\begingroup$ P.S. Does what I did here have a name? Is it a common filter type? $\endgroup$
    – endolith
    Apr 16, 2012 at 21:25
  • 1
    $\begingroup$ +1 - Great answer! About the automation of light source angle - you could use edge detector that computes both magnitude and gradient and then compute the weighted (by mag.) average of the gradient. The strongest responses should be in the direction of the illumination. $\endgroup$ Oct 5, 2012 at 18:33
11
$\begingroup$

First thing I would try is template matching, with templates rotated for all the angles with some step. Rotating template essential here. Also choice of template could be non-trivial - could be several with different lighting, and it could be blurred to allow for difference in shapes.

http://en.wikipedia.org/wiki/Template_matching#Template-based_matching_and_convolution

Next - HOG looks promising here. Another solution could be using strong corner detector like Moravec or Shi-Tomasi (with non-maximum suppression) and look for groups of 2-corners or 3-4 corners on the same line as candidates. After finding candidates you can apply active contour for verification (not sure if it would really help, but that is possibility)

http://en.wikipedia.org/wiki/Corner_detector

http://en.wikipedia.org/wiki/Active_contour

Yet another possibility is to use Hough transform for ellipses, possibly with not 2 but 3-4 free parameters.

$\endgroup$
7
$\begingroup$

Partial answer. Finding candidates with Mathematica:

p = ColorConvert[Import@"https://i.stack.imgur.com/38Ysw.jpg", 
    "GrayScale"] // ImageAdjust;
m = DeleteSmallComponents[Erosion[Dilation[DeleteSmallComponents[
      Binarize[EntropyFilter[p, 1] // ImageAdjust, .97], 10], 3], 5], 100];
ImageMultiply[Dilation[m, 3], p]

enter image description here

$\endgroup$
2
  • $\begingroup$ Interesting result... maybe combine with some other scheme... $\endgroup$
    – Matt M.
    Nov 22, 2011 at 4:11
  • $\begingroup$ @MAtt Yep I think that discarded at least 80% of the non targeted surface. After dilating the mask a bit, you should search for the ellipses.Irrespective the method you use (I am still thinking what I could do) it is far easier now you know the beasts are surrounded. $\endgroup$ Nov 22, 2011 at 4:37
1
$\begingroup$

I would start by using a sensitive edge detector (e. g. gradient magnitude with a low threshold), and then use the Hough transform to try to find the ellipses. Canny might still work as well. I am sure there are parameters you can tweak to make it more sensitive and pick up the blurred edges.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.