# Triangulation from set of points from left camera

in my program there are stereo cameras. The cameras were stereo calibrated. I have a set of 2D-Points from left image (for example, it could be some Harris Corners from left images). And the task is to estimate 3D position of this points with the reference to left camera. I started first with the estimation of corresponding points in image from right camera. I used OpticalFlow. Than this points were put in cvTriangulatePoints() function. But the estimated points in right camera image are not correct. My question is, is there some other method to get 3D position if I just know coordinates 2D points from left camera image? Thank you

You don't need to estimate points in the second image, you need to match them.

One way is to take a patch of pixels around the point in one image and see which of the points in the second image has a patch most "alike". One very simple way to do this is to take the sum of the squares of the difference between each pixel in the left-patch and the corresponding pixel in the right-patch.

If you could know the 3D position from only one camera, what would be the use of stereo vision?

You need to match two points found in both cameras, and then you can use epipolar geometry to approximate the 3D position.

For example your Harris Corners should have roughly the same features in both images to be a match.

• If you have objects of known shape and size in the plane it's perfectly possible to do 3D vision with a single camera. This is what most ART packages do with eg. square barcode markers – Martin Beckett Sep 12 '12 at 1:36
• Well that's a very specific case, which I would not call 3D vision anymore. – Geerten Sep 12 '12 at 6:51
• It works surprisingly well though! Given the inacuracies of aligning the two images you can often do better, at close range, with a target and POSIT – Martin Beckett Sep 12 '12 at 14:28
• Ok. I don't have experience with it, could be interesting. What is POSIT exactly? – Geerten Sep 12 '12 at 15:24
• Pose estimation = iterative algorithm, guess what an image of known sized square would look like from a particular direction, compare to image, repeat. It gives you the camera position/orientation from a single image of a known object – Martin Beckett Sep 12 '12 at 15:29