10
$\begingroup$

I've been sent here from this question in stackoverflow, please excuse me if the question comes too specific and it's not in the manners in here:)

The task is to find a glass with specific liquid in it. Let me show you the pictures and then describe what i'm trying to achieve and how i was trying to achieve so far in the description below the pictures.

The pictures: (seems i need at least 10 reputation to post pictures and links, so links will have to do :( otherwise you can look at the stack overflow question)

enter image description here

enter image description here

enter image description here

enter image description here

A detailed description: I was trying to implement an algorithm that would detect a glass of a specific shape in opencv (glass may be transformed by a different camera shot angle/distance). There will be also other glasses of other shapes. The glass i'm searching for will also be filled with some colored liquid that will distinguish it from glasses containing other colors.

So far, i have tried using SIFT feature extractor to try to find some features in the glass and then match them with other photos with the glass in it.

This approach worked only in very specific conditions where i would have glass in a very specific position and the background would be similar to the learning images. The problem also is that the glass is a 3d object and i don't know how to extract features from that (maybe multiple photos from different angles linked-somehow?).

Now i don't know what other approach could i use. I have found some clues on this (here https://stackoverflow.com/questions/10168686/algorithm-improvement-for-coca-cola-can-shape-recognition#answer-10219338 ) but the links seem to be broken.

Another problem would be to detect different "levels of emptiness" in such glass, but i haven't even been able to find the glass itself properly.

What would be your recommendations on the approach in this task? Would it be better to use a different way to find the local 3d object feature? Or would it be better to use other approach altogether? I have heard about algorithms "learning" the object from a set of multiple photos but i have never seen this in practice.

Any advice would be really appreciated

Improve this question
$\endgroup$
2
  • $\begingroup$ any final solution with full source code sample in C# ? $\endgroup$
    – Kiquenet
    Apr 14, 2014 at 20:24
  • $\begingroup$ wow that's quite a specific request. the problem does not have a solution yet and i won't be investing more time in solving it. i believe, judging by the paper mentioned in here, that the scientific grounds won't have a solution very soon neither, as the suggested algorithm had very poor precision rates. anyways, since this project was for my job, i ended up argumenting about a compromise with a client, as the task is unreal to complete nowdays. used some regular haar-like feature detectors for "anything that looks like a cup" and then selected yellow hues to detect beer. not the original task $\endgroup$
    – user1916182
    Apr 15, 2014 at 22:17

3 Answers 3

Reset to default
2
$\begingroup$

The paper referenced in your link seems to be this one.

Of particular interest there is Table 1 (included below). The accuracy rates aren't great, though they are better than other approaches.

enter image description here

Improve this answer
$\endgroup$
0
$\begingroup$

Maybe this paper can help you: http://ai.stanford.edu/~ang/papers/iros09-ScalableLearningObjectDetectionGPU.pdf

Although they use the active stereo system in addition to 2D images in order to acquire depth images, is interesting how they use the patch-based features, constructing a dictionary of the object with many small fragments and then training a classifier. Maybe you can add this features to improve your detection rate.

Improve this answer
$\endgroup$
1
  • $\begingroup$ i don't think this work can be used very well in wild praxis. the robot described in the paper requires a depth sensor to detect depth information... not very practical to carry a kinect around when taking random pictures on a cellphone or something... but yeah, the patch based features are very interesting approach! $\endgroup$
    – user1916182
    Feb 14, 2016 at 15:09
-1
$\begingroup$

There has been much work done on this when it comes to facial recognition software. For example, if you notice on Facebook when tagging photos the location of the faces are boxed and suggested to you.

I have seen a lot of literature on facial recognition in images using neural networks and a quick Google search will undoubtedly turn up a vast amount of information on the subject. These networks take the pixels of the image as inputs. In your case, the way the opacity changes/light reflects off the glass may be good identifying features that the network will learn.

One problem may be the number of photos you have to use as training data and the pre-processing of these (i.e. identifying the faces yourself). If it is unfeasible to do this for enough images to train your network well enough then you will have to look for some shortcuts in the learning stage. This paper is relevant to what you want to do: http://www.ll.mit.edu/publications/journal/pdf/vol04_no2/4.2.5.neuralnetwork.pdf

Fortunately this is a very active field and much of the code necessary for this type of problem is readily available online.

Once you are able to identify the glasses in the images you can perform further analysis from there.

Improve this answer
$\endgroup$
2
  • $\begingroup$ this is completely irrelevant $\endgroup$
    – user1916182
    Jan 7, 2018 at 2:13
  • $\begingroup$ Interesting you should say that. Anyway here is a link to the detection algorithms used by Facebook based on neural networks. Good luck, github.com/facebookresearch/Detectron. $\endgroup$
    – rwolst
    Jan 23, 2018 at 9:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.