Segmentation of half transparent material, e.g. glass

Question

I'm totally stuck on an issue regarding the segmentation of glassy objects. I need to get the object as precise as possible. My approaches were different. At first I tried to remove the background, so that only some sharp contours are left. But that only works for objects which have sharp edges / gradients. Otherwise the object itself is also removed. I posted two different images.

I tried to remove the background via morphological operations, like grayscale dilatation and a divison on it. but it didnt help much. after it, I tried a k-means with k=3 for getting the modified background separated from the gray and black values of the glass. That wasn't successfull in some cases, but not overall/in average. I also tried to make a canny edge detection with an overall blured filter, but that lead to weaker results in form of open contours, a lot of noise, etc. pp.

Canny with automatic threshold results:

testimg = imread('https://i.stack.imgur.com/Y6YO0.png');  
imshow(testimg)
imedges = edge(testimg,'canny');
imshow(imedges);

Same goes for the second image.

As you can see, there is a lot of noise inside and outside and doubled edges from the glas border. Even there are gaps in the edges.

So, I need your advices for getting a general approach for dealing with this problem of half-transparent materials, not for just these two images.

1) Other ideas for removing the background without damaging the object?

2) Other segmentation methods for getting the object separated from the background?

If it's possible, then with Matlab, IPT or statistical toolbox hints. Any other hints are also welcome!

Thank you for your answer in advance. Sincerely

Answer 1

Why not just use a simple 2D FFT (guassian) high pass filer?

I did this real quick using MATLAB

Shard #1 using high pass FFT:

The same thing is done on #2.

Shard #2 using high pass FFT:

As you can see, the background and glass area is wiped out, and only the edges are traced. I did not spend any time on it, but you can threshold the HP filtered output to have more crisp edges, or push the HP cut off higher.

Is this more the results you are looking to get?

Answer 2

This is no attempt to answer the whole question, but I do have an idea about "cleaning the image".

You said you tried morphological operations already, and this is a variation to the idea, hopefully an upgrade.

This article: A. Vichik, R. Keshet, D. Malah: Self-dual morphology on tree semilattices and applications proposes a way to enhance on the classical morphological operators in a way that can add more desirable properties to them.

The article suggests to choose a hierarchical representation of an image according to desirable properties, and then proposes a method to define operators such as erosion, dilation, opening, top-hat on that representation. In their own words:

We have presented a general framework for producing new morphological operators (...)

I explained these hierarchical, tree-shaped structures in the second part of this answer (Semantic approaches), to which you can add Extrema-Watershed Tree mentioned in the article I linked here (and again).

It is an upgrade to (quoting the authors) "traditional grayscale mathematical morphology" because the operations keep the desirable properties of the representations. E.g. if your hierarchical representation is self-dual, your operators will be really self-dual (e.g. compare with quasi-self-dual opening-closing by reconstruction which is not really self-dual.)

The linked article also presents some results in filtering out the noise - you can compare their results from the article (and from the Thesis referenced in the article) to what you need (at least visually) and see if it would work for you before starting to code.

So, while choosing the simplest representation (max-/min-) tree will yield exactly the classical operations, choosing a self-dual tree which best suits your needs might give you a robust enough approach.