0
$\begingroup$

I'm trying to set-up a machine vision application in which I have to detect the position of a printing nozzle (in the red sqaure in the image) in a field of view in order to prcisely position a ganty stage.

I currently struggle to detect the wished features. I have tried to implement a Haar cascade using opencv but the detection is not performing.Image from the camera witht eh region of interest in red

To train my classifier I used the full image with the location of the nozzles blacked out as negative and the cutout of the red rectangle as positives. I have few positives and negatives (15 of each) that are basically the same image with different illuminations/expositions and placements of the printer head in the field of view.

Since this approach did not work, what kind of image recognition should I use in that case? Would a training of a custom set in tensorFlow give better results? I've seen implementations of DNN using existing vast libraries of tagged images but I'm not sure how if this is the best method for this application.

Thanks in advance

$\endgroup$
  • $\begingroup$ Is the printing head expected to be at the same distance from the camera (z dim) but in any (x,y) within its FOV? $\endgroup$ – A_A Mar 18 at 13:32
  • $\begingroup$ The head is moving in a XY plane over the camera. So in the edge of the FOV the printhead will be further (direct distance) from the center of the lense but the XY plane is at constant Z-distance of the camera. $\endgroup$ – EvilGarfield Mar 18 at 17:18
  • $\begingroup$ Would it be possible to post a picture with some representative background please? I am assuming that the one in the post now is "empty" (?) When this becomes operational, would there be any clutter in the background or is this more or less what the camera would be looking at on average (?), but with the print heads moved to different locations maybe. $\endgroup$ – A_A Mar 18 at 23:58
  • $\begingroup$ The camera has a shallow depth of focus. This is basically what the background will look like, $\endgroup$ – EvilGarfield Mar 19 at 6:21
1
$\begingroup$

If the camera has a shallow depth of focus and the background is mostly blurry and the foreground is only moving at one focal plane, why not use a simpler method, such as Template Matching?

Since the printing head is changing its angle of view with respect to the lens as it moves in the $x,y$ dimensions, you might have to use more than one templates but overall the process is straightforward: Apply the cross correlation in a sliding window across the image and stop where the cross correlation peaks.

This example is in MATLAB but easily transferable to other platforms.

This sliding window might seem slow but is unavoidable. Neural networks do it implicitly at the lowest level of inputs. The good news here is that the printing head image does not move in the $z$ dimension, in which case the scale would change too. This would mean that you would have to repeat this process for the scales of interest.

You might be able to derive "hints" of where the printing head is roughly in the image, by tracking its silhouette. If you threshold the image for example, you might want to look for a distinct pattern of large rectangular areas in black, that roughly match the size and shape of the arm. This would help in making the search area where the normalised cross correlation is applied, smaller. But overall, I would not expect a modern Desktop or even SBC to struggle with an HD image and a patch even as big as 128x128 pixels.

Hope this helps

EDIT:

In terms of a pesudo-code instruction:

  1. Take a representative image of the nozzle (don't crop it too much around the nozzle but make sure the bit you want to recognise is clearly visible).

  2. Given an image from the monitoring camera, run a normalised cross correlation across the image. How you are going to do this depends on the platform you are using but MATLAB, Octave, Python all have means of running a normalised cross correlation, fast, in this way

  3. The output of this will be a Real two dimensional array that will have a single peak exacty at the point where the template had the best match.

  4. Threshold that match and take the centroid of the region (if it spans more than one pixel) to find out where the nozzle is in the camera image.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for your help. So if I understood correctly, I need to take an image where the nozzle is exactly centered with my camera (as I want it to be) and then compute the crosscorrelation between this image and live ones to see how much the nozzle is displaced in the live ones? I will try to implement that method and keep you updated. $\endgroup$ – EvilGarfield Mar 19 at 15:11
  • $\begingroup$ @EvilGarfield Glad to hear this was helpful. Please see updated response. $\endgroup$ – A_A Mar 19 at 15:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.