Take the 2-minute tour ×
Signal Processing Stack Exchange is a question and answer site for practitioners of the art and science of signal, image and video processing. It's 100% free, no registration required.

Hi CV/Pattern Recognition Community,

I've got a serious problem regarding the segmentation of an image. The scenario is an atmosphere within a furnace which makes my head go insane. And I need to detect object contours of different materials (glass, ceramics, Al, Ir,..) in a short period of time (<10 seconds) and not just for one special case. I also need the contour in a sequential row of pixels for the code. Therefore a chain code or so called border/contour following is also needed, so open holes aren't good. In the background are non linear noises, approximately of dust, particles or somewhat else, that are appearing from time to time.

Matlab or OpenCV suggestions are welcome.

To make it more clear, I've posted another image of my goal and a half-transparent object, which needs also to be detected. Also further examples which need to be aware of. example1 example2 example3 example4

As you can see in the Image #1, there are particles in the right part of the image and nearby the outer contour of the star, which is the object. Also the overall contrast is not very good. The object itself stands on an underground, which is not relevant for the contour detection. The image #2 shows a halftransparent object, which is also possible.

I want to find contour/perimeter of that object, like on the next screen (red line). The two rectangles (yellow) are marking the starting (left) and the ending point (right). The blue line is ignorable. example2

At first I thought that I could solve the problem of that filthy atmosphere with just filters. But after a honourable ammount of investing time, I just realized, that I have to elimate or reduce the noises significantly in order to increase contrast of foreground and background. I'd tried a lot of methods, like histogram equalization, Otsu-adaptive equalization, linear filters (e.g. gauss), nonlinear filters (median, diffusion), Active Contours, k-Means, Fuzzy-c-means and also Canny for pure Edge Detection in combination with morphological operators.

  • Canny: The particles and the atmosphere are causing holes, but I need a complete contour of the object. Still with closing, dilatating of morphological operators it is not well enough. Canny has still the best results of all methods I've studied because of hysteresis.
  • Active Contours: They work on edges/gradients as well, they act completely crazy after initializing inside the object, which maybe is caused by the edge map resulting the 'open' object. As far as I know the contour has to be closed. Tried it with different derivates (GVF/VFC/Classic Snake).
  • k-Means: Results include the furnace atmosphere, because of foggy background. Same for fuzzy-c-means. I chose two clusters, because of separating the object from the background. More clusters lead to weaker results.
  • Histogram/Otsu: Because of the very close gray intensities (imho!), it's merging the object with the background. Tried it with local and global methods.
  • Filters: Especially GLPF or other LPF are smearing the edges, which is not so good and doesn't even reduce the foggy atmosphere.
  • Non-Linear Filters are preserving the edges. Most of them take too long for calculating the large images. Took a fast bilateral filter for now. Results see below.

Therefore not a single method is good enough for post-processing steps, because the gained results of the object segment are poor competed to an existing algorithm. That existing algorithm is very local and therefore it works for this very special scenario.

So I am asking you, if I have missed something completely... I have no further idea how to process and how I should get good contour results, without having gaps or holes.. Is it possible without making a lot of changes on the CCD and the physical environment? Thanks in advance!

Last Approach so far (after a long night of experiments with MOs):

  • Bilateral Filter (edge preserving, but smoothing homogenous areas)
  • Canny (Sigma = 2, Threshold = [0.04 0.08])
  • Morphological Operations (MO): bwareopen,closing,remove & bridge
  • bwlabel for selecting only the perimeter of the contour, which removes unwanted noises. no updated screenshots yet, but it works for the star. the glass gots an inner-contour which is connected to the outter contour, which can also be seen on the screenshot below.

So I'm afraid that I need a special algorithm for the traversal of the outter contour. It will be some clockwise/counterclockwise lookup of neighbourhood. That clockwise/counterclockwise step can switch, if there is a corner point. if there is a gap, increase the radius and look again. if there are two or more possible following points, take the one who got the same direction as the previous. Do you think, that contour following algorithm makes sense?

Edges of Glass Star

share|improve this question

migrated from stackoverflow.com Oct 12 '12 at 16:34

This question came from our site for professional and enthusiast programmers.

    
Have you tried adaptive threshold? You don't seem to mention it. I think OTSU should work somehow after removing noise but maybe adaptive threshold is better. –  Rui Marques Oct 12 '12 at 17:17
    
Hi Rui, I tried adaptive thresholding with this Matlab Extension: Adaptive Thresholding Here are the results for playing around with the window parameters: 1st Try 2nd Try 3rd Try As you can see, with increasing parameters the middle part goes from white to black (what is good, imho) but the rest of the background also turns into black, what is bad. –  mchlfchr Oct 12 '12 at 18:09
3  
Have you tried some basic image reduction techniques? Such as subtracting a flat-field correction to get rid of that darkening on the top right (en.wikipedia.org/wiki/Flat-field_correction). Also, if the particles are static, they would instantly be removed. Then, you could use any edge detection method you want... –  PhilMacKay Oct 12 '12 at 18:25
    
Hi Phil, as far as I am concerned and as I know, there is a series of pictures who are shot before an object gets into the furnace. Kind of calibration is therefore here. I'll talk to the physicist who is responsible for the CCD and the environment on monday. But thanks for the advice, I'll give it a shot! –  mchlfchr Oct 12 '12 at 19:03
    
The second image you added looks totally different. Can you post all possible images? –  Andrey Oct 14 '12 at 13:04

6 Answers 6

You can try the following:

share|improve this answer
    
Hi oli, regarding sparse methods: could you be more specific of what methods of that code I should use? I'm not very deep into that section and I didn't find something helpful in the docs regarding de-noising or blurring... Thank you in advance. –  mchlfchr Oct 12 '12 at 17:28
1  
You can find an "easier to use" version, there: lear.inrialpes.fr/people/mairal/denoise_ICCV09.tar.gz –  oli Oct 12 '12 at 17:36
    
sorry to complain another time ;-) ... do you have win32 sources as well? thank you again! –  mchlfchr Oct 12 '12 at 17:54
    
I am affraid I do not... –  oli Oct 12 '12 at 17:58

I think that you gave up on threshold techniques too early. Take a look at your histogram, it is clearly tri-modal: (I removed the white columns to the right of your image manually, I assume that they are not part of the image - please take this image before running my code)

enter image description here

Take a look at all values in the first group:

enter image description here

In order to find modes in tri-modal histogram, it is possible to use K-means clustering with K=3 on intensity. The following Matlab code finds th1=67 on your code. The idea is to assume that you have the 3 sets, and calculate the weighted centroid on each one. Then, each intensity level is assigned to its own cluster. You stop when the weighted centroids cease moving. Here is the result of finding two thresholds on your image, shown on the histogram.

enter image description here

function [th1,th2]=SegmentHistTo3()
    im = imread('http://i.stack.imgur.com/U2sc5.png');
    h = imhist(im(:,:,1)); %# Calculate histogram

    th1new = round(256/3); %# Initial thresholds
    th2new = round(256*2/3);
    th1 = 0;
    th2 = 0;

    while (th1~=th1new) || (th2~=th2new) %# While the centroids keep on moving
        th1 = th1new;
        th2 = th2new;

        wa1 = WeightedAverage(h,1,th1);  %# Calculate 3 weighted averages
        wa2 = WeightedAverage(h,th1+1,th2);
        wa3 = WeightedAverage(h,th2,numel(h));

        th1new = round( (wa1+wa2)/2 );  %# The thresholds are middle points between the averages
        th2new = round( (wa2+wa3)/2 );
    end

    figure; hist( double( reshape(im(:,:,1),1,[]) ),256);
    hold on;
    plot( [th1 th1],[0 max(h)],'r','LineWidth',2);
    plot( [th2 th2],[0 max(h)],'r','LineWidth',2);

    figure;imshow( im(:,:,1)<th1);
end

function wa = WeightedAverage(region,th1,th2)    
    regionNonEmpty(th1:th2) = region(th1:th2);
    wa = sum( regionNonEmpty .* (1:numel(regionNonEmpty))) / sum(regionNonEmpty);    
end

Solving the problem afterwards is a piece of cake, simply do some simple morphological operations, like opening.

share|improve this answer
1  
Hi Andrey, but how should I do a generalization of that thresholding you mentioned? I got several cases, not just that one and I still need automation. And the Otsu Thresholding (function in matlab) gave me no good results. Any further hints? Kind regards –  mchlfchr Oct 13 '12 at 15:06
    
Hi again, thanks so far, but the code doesn't work. Blank figure screen appears. Tried it with my original data (bitmaps) and the PNG you posted above. I am debugging meanwhile... –  mchlfchr Oct 13 '12 at 15:54
    
@mchlfchr, do you have image processing toolbox? If you don't, it is possible to change imhist to hist –  Andrey Oct 13 '12 at 15:56
    
@mchlfchr, please see updated version –  Andrey Oct 13 '12 at 15:58
    
Andrey, if I insert the original bitmap file, the results are as I mentioned in my origin post. Maybe this is happening regarding the resolution? Source image is 576x768 pixels and grayscale (256). Here are the results, if I use your function with my original image: i.imgur.com/UXALJ.png histogram-figure of your function: i.imgur.com/7RiPP.png Thanks for your help! regards –  mchlfchr Oct 13 '12 at 17:43

As suggested above, thresholding can be very effective on this image, which is essentially binary, except that a constant threshold will not do because of uneven lighting. You need adaptive thresholding.

My advice would be to do background reconstruction with a simple model (possibly planar [3 DOF] or quadradic [6 DOF]), by sampling a small number of values in the light regions. The best is to use small ROIs to average away the noise. Then correct the shading by subtracting (or dividing by) the background values.

If human interaction is not an option, you can automate the search for background areas by first straight Otsu and considering uniform ROIs (low variance) well under the threshold. After a first background reconstruction, you can probably improve by applying this process to the flat-corrected image.

The whole process can be implemented to run well below a second.

share|improve this answer
    
Hi Yves, an automated processing is prefered. The DOF aspect is interesting, but I'm not sure about the Otsu method, because Otsu itself is not working well. Am I understanding it right that you want to pick random areas of the image and then you would threshold after mean value over all picked areas? Kind regards –  mchlfchr Oct 13 '12 at 12:52

I think best way is to use Active contours. If you are not aware what active contours are have a look at this video on youtube http://www.youtube.com/watch?v=ijNe7f3QVdA

Basically, u need to give an initialize segmentation and it will improve the shape. My suggestion is to one of the methods discussed on this post and use active contours as a 2nd step ie. as an improvement step.

Here is an implementation of active contours you could use http://www.mathworks.com/matlabcentral/fileexchange/19567

share|improve this answer
    
Welcome to dsp.se :) Thanks for contributing, you provided a nice answer. If you want to make it even better, I think it would be interesting to provide answers to some of these questions: Why do you think this is the best approach (e.g. do you have personal experience with the tecnhique)? Which of the already suggested approaches do you think would work well in combination with your suggestion? Offer a short explanation of the technique, or, if you have time, try to provide experimental results using the technique on the example images provided. And have fun on dsp! –  penelope Oct 26 '12 at 16:13
    
@mkuse, as you may have read the initial post, i already had tried the active contours, combined with noise reduction and edge maps. the results were bad and had a bad run-time for large images. –  mchlfchr Oct 27 '12 at 6:50
    
how about you have a look at noise reduction techniques. You can find a summary of those here : lnmiitdip.files.wordpress.com/2011/12/… –  mkuse Oct 27 '12 at 11:04
1  
@mkuse, I already mentioned the mechanics you have posted in your PPT file in my initial post. I edited my initial post, to make it more clear, what kind of filters I've used. –  mchlfchr Oct 27 '12 at 13:08

You clearly know what you're about but you haven't mentioned using thresholding, in particular have you tried applying a global threshold using Otsu to calculate the right level, then finding contours and selecting the largest ?

[Edit to clarify]

Global threshold obviously won't work due to the visible graident across the image.

I had a quick play with this and find that if you break the image into 6 chunks (2 rows of 3 columns equal sized) , then perform thresholding using Otsu on each one and then reassemble, it does a pretty god job at cleaning up the image.

There are still some minor artifacts on the upper right portion of the star.

It occurs to me that since the object has straight line boundaries, you might want to consider a Hough transform to extract these edges, intersect them to locate vertices and use the result as your object contour.

share|improve this answer
    
Hi Dave, I tried Otsu, but it brings the effect that the right upper background is merging with the object, which is absolutely inacceptable. –  mchlfchr Oct 12 '12 at 15:16
    
Hi Dave, Hough is not an option, because of run-time requirements and as far as I have knowledge about the HT, it's very time consuming for large images. –  mchlfchr Oct 13 '12 at 15:55

Are the outlines always straight lines or known curves?

If so then rather than trying to get each pixel along the edge correct I would use Hough transforms to get the equations of the lines and then recreate the countours from the lines and itnersections

share|improve this answer
1  
As I already mentioned: I need nearby real-time approaches. And as far as I know the HT, it's very time consuming. Another aspect is, that I don't know the curves and the lines aren't always straight. The contour depends on the material, which is in the furnace (for more information, see my origin post). –  mchlfchr Oct 13 '12 at 17:27
    
For straight lines it's pretty fast, and if you know roughly where the lines are (eg from a previous frame) you can search only that paramter space –  Martin Beckett Oct 13 '12 at 18:36

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.