Image segmentation issue of different materials

Question

Hi CV/Pattern Recognition Community,

I've got a serious problem regarding the segmentation of an image. The scenario is an atmosphere within a furnace which makes my head go insane. And I need to detect object contours of different materials (glass, ceramics, Al, Ir,..) in a short period of time (<10 seconds) and not just for one special case. I also need the contour in a sequential row of pixels for the code. Therefore a chain code or so called border/contour following is also needed, so open holes aren't good. In the background are non linear noises, approximately of dust, particles or somewhat else, that are appearing from time to time.

Matlab or OpenCV suggestions are welcome.

To make it more clear, I've posted another image of my goal and a half-transparent object, which needs also to be detected. Also further examples which need to be aware of.

As you can see in the Image #1, there are particles in the right part of the image and nearby the outer contour of the star, which is the object. Also the overall contrast is not very good. The object itself stands on an underground, which is not relevant for the contour detection. The image #2 shows a halftransparent object, which is also possible.

I want to find contour/perimeter of that object, like on the next screen (red line). The two rectangles (yellow) are marking the starting (left) and the ending point (right). The blue line is ignorable.

At first I thought that I could solve the problem of that filthy atmosphere with just filters. But after a honourable ammount of investing time, I just realized, that I have to elimate or reduce the noises significantly in order to increase contrast of foreground and background. I'd tried a lot of methods, like histogram equalization, Otsu-adaptive equalization, linear filters (e.g. gauss), nonlinear filters (median, diffusion), Active Contours, k-Means, Fuzzy-c-means and also Canny for pure Edge Detection in combination with morphological operators.

• Canny: The particles and the atmosphere are causing holes, but I need a complete contour of the object. Still with closing, dilatating of morphological operators it is not well enough. Canny has still the best results of all methods I've studied because of hysteresis.
• Active Contours: They work on edges/gradients as well, they act completely crazy after initializing inside the object, which maybe is caused by the edge map resulting the 'open' object. As far as I know the contour has to be closed. Tried it with different derivates (GVF/VFC/Classic Snake).
• k-Means: Results include the furnace atmosphere, because of foggy background. Same for fuzzy-c-means. I chose two clusters, because of separating the object from the background. More clusters lead to weaker results.
• Histogram/Otsu: Because of the very close gray intensities (imho!), it's merging the object with the background. Tried it with local and global methods.
• Filters: Especially GLPF or other LPF are smearing the edges, which is not so good and doesn't even reduce the foggy atmosphere.
• Non-Linear Filters are preserving the edges. Most of them take too long for calculating the large images. Took a fast bilateral filter for now. Results see below.

Therefore not a single method is good enough for post-processing steps, because the gained results of the object segment are poor competed to an existing algorithm. That existing algorithm is very local and therefore it works for this very special scenario.

So I am asking you, if I have missed something completely... I have no further idea how to process and how I should get good contour results, without having gaps or holes.. Is it possible without making a lot of changes on the CCD and the physical environment? Thanks in advance!

Last Approach so far (after a long night of experiments with MOs):

• Bilateral Filter (edge preserving, but smoothing homogenous areas)
• Canny (Sigma = 2, Threshold = [0.04 0.08])
• Morphological Operations (MO): bwareopen,closing,remove & bridge
• bwlabel for selecting only the perimeter of the contour, which removes unwanted noises. no updated screenshots yet, but it works for the star. the glass gots an inner-contour which is connected to the outter contour, which can also be seen on the screenshot below.

So I'm afraid that I need a special algorithm for the traversal of the outter contour. It will be some clockwise/counterclockwise lookup of neighbourhood. That clockwise/counterclockwise step can switch, if there is a corner point. if there is a gap, increase the radius and look again. if there are two or more possible following points, take the one who got the same direction as the previous. Do you think, that contour following algorithm makes sense?

Answer 1

You can try the following:

• Efficient Graph-Based Image Segmentation: http://www.cs.brown.edu/~pff/segment/ (code available)

• GraphCut-based Segmentation: http://www.csd.uwo.ca/~olga/OldCode.html (code available)

• Denoise the image first using sparse methods: http://spams-devel.gforge.inria.fr/ (code available)

Answer 2

I think that you gave up on threshold techniques too early. Take a look at your histogram, it is clearly tri-modal: (I removed the white columns to the right of your image manually, I assume that they are not part of the image - please take this image before running my code)

Take a look at all values in the first group:

In order to find modes in tri-modal histogram, it is possible to use K-means clustering with K=3 on intensity. The following Matlab code finds th1=67 on your code. The idea is to assume that you have the 3 sets, and calculate the weighted centroid on each one. Then, each intensity level is assigned to its own cluster. You stop when the weighted centroids cease moving. Here is the result of finding two thresholds on your image, shown on the histogram.

function [th1,th2]=SegmentHistTo3()
    im = imread('https://i.sstatic.net/U2sc5.png');
    h = imhist(im(:,:,1)); %# Calculate histogram

    th1new = round(256/3); %# Initial thresholds
    th2new = round(256*2/3);
    th1 = 0;
    th2 = 0;

    while (th1~=th1new) || (th2~=th2new) %# While the centroids keep on moving
        th1 = th1new;
        th2 = th2new;

        wa1 = WeightedAverage(h,1,th1);  %# Calculate 3 weighted averages
        wa2 = WeightedAverage(h,th1+1,th2);
        wa3 = WeightedAverage(h,th2,numel(h));

        th1new = round( (wa1+wa2)/2 );  %# The thresholds are middle points between the averages
        th2new = round( (wa2+wa3)/2 );
    end

    figure; hist( double( reshape(im(:,:,1),1,[]) ),256);
    hold on;
    plot( [th1 th1],[0 max(h)],'r','LineWidth',2);
    plot( [th2 th2],[0 max(h)],'r','LineWidth',2);

    figure;imshow( im(:,:,1)<th1);
end

function wa = WeightedAverage(region,th1,th2)    
    regionNonEmpty(th1:th2) = region(th1:th2);
    wa = sum( regionNonEmpty .* (1:numel(regionNonEmpty))) / sum(regionNonEmpty);    
end

Solving the problem afterwards is a piece of cake, simply do some simple morphological operations, like opening.

Answer 3

As suggested above, thresholding can be very effective on this image, which is essentially binary, except that a constant threshold will not do because of uneven lighting. You need adaptive thresholding.

My advice would be to do background reconstruction with a simple model (possibly planar [3 DOF] or quadradic [6 DOF]), by sampling a small number of values in the light regions. The best is to use small ROIs to average away the noise. Then correct the shading by subtracting (or dividing by) the background values.

If human interaction is not an option, you can automate the search for background areas by first straight Otsu and considering uniform ROIs (low variance) well under the threshold. After a first background reconstruction, you can probably improve by applying this process to the flat-corrected image.

The whole process can be implemented to run well below a second.

Answer 4

I think best way is to use Active contours. If you are not aware what active contours are have a look at this video on youtube http://www.youtube.com/watch?v=ijNe7f3QVdA

Basically, u need to give an initialize segmentation and it will improve the shape. My suggestion is to one of the methods discussed on this post and use active contours as a 2nd step ie. as an improvement step.

Here is an implementation of active contours you could use http://www.mathworks.com/matlabcentral/fileexchange/19567

Answer 5

You clearly know what you're about but you haven't mentioned using thresholding, in particular have you tried applying a global threshold using Otsu to calculate the right level, then finding contours and selecting the largest ?

[Edit to clarify]

Global threshold obviously won't work due to the visible graident across the image.

I had a quick play with this and find that if you break the image into 6 chunks (2 rows of 3 columns equal sized) , then perform thresholding using Otsu on each one and then reassemble, it does a pretty god job at cleaning up the image.

There are still some minor artifacts on the upper right portion of the star.

It occurs to me that since the object has straight line boundaries, you might want to consider a Hough transform to extract these edges, intersect them to locate vertices and use the result as your object contour.

Answer 6

Are the outlines always straight lines or known curves?

If so then rather than trying to get each pixel along the edge correct I would use Hough transforms to get the equations of the lines and then recreate the countours from the lines and itnersections