2
$\begingroup$

Problem: I have a photo of an object (a manufactured part like the attached photo below), using my Andoird phone camera I want to verify if the object in camera preview matches to the template or not. (in other words, is it the same part as the template or not)

  • I can make the user to move the camera in order to have similar view of the template in camera preview as the template however there will be different noise level and/or lighting and maybe different background.

Question: What do you recommend me to use for solving this problem? I was thinking of Canny edge extraction and then matching the camera frames towards the canny edge extract from template? is this a good idea? if yes would you please tell me how can I implement this? any resources? samples? (I can do the Canny edge extraction but couldn't find a way to do the matching)

if Not a good idea then what do you recommend?

Things I have tried:

  1. Feature Extract and Matching: I used few different extractor and matcher implementations from OpenCV and my app is working and drawing the detected feature points and matches, etc. however being a beginner with image processing I cannot make sense of the result and also how to know what is a match. any idea, help, good resources?
  2. Template Matching: I used OpenCV template matching however the performance was horrible and I decided that this cannot be the solution.

sample template photo

$\endgroup$
  • $\begingroup$ ` used OpenCV template matching` What did you use? E.g. ORB should be fast and accurate. $\endgroup$ – FindOutIslamNow Apr 17 '18 at 12:09
2
$\begingroup$

That's hard. I'm not even sure I can do this myself before I actually try. It is definitely not something that can be described in a DSP.SE answer.

About canny edge even if it is precise you will just get an image like a line drawing as an output. The point is what's next. There are many things to try the are many theory about pattern matching from the edge but it not that easy.

I will try something easier before going that path for example I will try to locate coordination and radius of those circles. (nuts, holes, etc) And see if I can identify the part using just those info. Then I'm going for the straight lines.

Please note that I know just an overview of theories in this field and not generally do image processing but you can try what I suggested using OpenCV for sure.

$\endgroup$
  • $\begingroup$ Thanks for the reply. About Canny edge my idea is to have the template same size of the camera inout (maybe reduce the camera resolution) and then use template matching between the two which should be fairly fast because it's just matching one time for every frame. What do you think? however I need to find out how can I adjust the sensitivity and also if it's possible at all or not. $\endgroup$ – Asha Aug 18 '14 at 19:00
  • $\begingroup$ @Asha It would indeed be fast with template matcher but the success would be really low. For the different lighting, angles and background, using shape/contour matching would have more success. $\endgroup$ – Tyathalae Aug 19 '14 at 8:35
  • $\begingroup$ @SelimArikan: Is it possible to give me more details on the shape/contour matching? any references? sample codes? thanks $\endgroup$ – Asha Aug 19 '14 at 9:29
1
$\begingroup$

Okay, Feature Extract and Matching is the way to go for a beginner. Other methods such as relative pose estimation, triangulation, 3d matching etc. would be much more complex.

Your object is metallic and thus shiny. Also, without correct lighting it is quite difficult to match features. But still, as much as I could see from that picture, you have plentiful number of features out there. Extract your descriptors and match them using FLANN. If you use Sift features, it will provide you with enough perspective invariance.

I see that most of your template is planar. For that reason, after computing the matches with Sift+FLANN, you might as well go for a homography computation using RANSAC. Because RANSAC could tolerate significant number of outliers (~50%), you will end up with a nice homography. Luckily, OpenCV has that too. Of course, this homography will not always be a very good one and you could detect this, with something like:

bool validate_homography(const double h[9]) { const double det = h[0] * h[4] - h[3] * h[1]; if (det < 0) return false; const double N1 = sqrt(h[0] * h[0] + h[3] * h[3]); if (N1 > 4 || N1 < 0.1) return false; const double N2 = sqrt(h[1] * h[1] + h[4] * h[4]); if (N2 > 4 || N2 < 0.1) return false; const double N3 = sqrt(h[6] * h[6] + h[7] * h[7]); if (N3 > 0.002) return false; return true; }

If this returns true, you could accept your match.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.