Measuring non-planar objects in image

Question

I'm working on a computer vision application (using openCV) and the goal is to measure the width of an object before/after it falls on a container. So after the fall its position will be random, something like this to be clear:

The only object I want to measure will be clear filtering the image and will be always on top, but its width won't be parallel to the image plane of the camera in the general case. In the worst scenario the longer face could be very occluded, getting things very hard.

So my question is: what will lead to better accuracy between these 2 strategies?

1. Getting a pair of stereo images of the container, locate two points on the longer face of the object (vertex or medium point of the borders) and then calculate the distance in 3D space between these points.

2. Working with a single camera close to the object before it falls into the container, where its motion will be perpendicular to the camera so the entire width will be on a plane parallel to the image plane. Using a known size reference (at the same distance of the object to be measured) and calibrated camera will lead to width measure from a single image simply counting pixels.

Additional info: the distance between camera and object will be 2 to 2.5 m and I need 10mm accuracy

The second path sounds cheaper, easier to implement and likely more accurate..but I'm open to suggestions or other ideas.

Answer 1

Stay away from the stereo solution, it will be a mess.

2D measurement using edge fitting is by far easier and more accurate. Make sure that the objects are contrasted wrt the background, and that only the top face is visible.

Answer 2

I would go for option a combined approach, which requires a stereo setup:

First of all, 3D depth information (which you would use as an additional cue) can be obtained through a stereo matching method. Depth images mean that you end up with 3D surfaces. At this point you could use a surface matching algorithm to initialize the positions of these objects (or cubes). I have implemented such an algorithm in OpenCV. You could find this here.

Luckily, for a calibrated stereo setup, you will have the intensity images and can use those as well. By now, you have the initial poses of the cubes and you would like to refine them. I would then suggest that you use a projective version of the ICP (iterative closest point) algorithm to obtain the most accurate fitting, to the image edges. Again, you could use both cameras in your error function (which you minimize) to reduce noise.

Note that, the stereo images may suffer from extrinsic calibration errors, as well as measurement errors, decreasing the overall accuracy. That's why you use it to initialize. For cases which do not necessitate accuracy you could stop at this point. If your conditions are as strict as you mention, then simply, the monocular/edge based techniques (the second stage, I described) will provide you a better result.

Answer 3

Dream job: QC at a caramel factory. Yummmmm.

The pic looks like caramels, but then the 10mm spec makes no sense. This question is three years old and the OP hasn't been around for nearly that long, but it would be nice to find out what solution was actually implemented, and perhaps what is actually getting measured.

Solution #2 is undoubtedly superior. Especially if the objects have a very consistent orientation and flight path as they drop.

I have an idea for superior accuracy in the case where there is a little variation in the distance to the camera, and/or, the tilt of the plane being measured. Four pairs of lasers. Each pair acting like the altimeter spotlights in the movie "The Dam Busters", only aimed so the focal point is a little behind the surface being measured so the dots are a small distance apart. The pairs are then aimed at the plane to be measured near the corners. With a little calibration, the distance between the dots can be used to very accurately locate the plane in 3D space, thus allowing the measurement from image pixel width to be tweaked to a very high degree of precision.

If only a limited portion of the field of vision is required for the object there is another trick that works: Use a mirror. A small mirror can be place near the camera so it gets binocular vision, or a larger mirror can be used next to, or behind, the object. With a little trigonometry and calibration, 3D location measurments can be made. This beats two cameras in the case of a moving object as it eliminates the need for synchronization.

Ced