I got 2 pictures which is almost parallel, and not positioned very far from each other.
I'm using OpenCV to try to create a disparity map (Stereo correspondence).
Because I'm trying to use it in a real world scenario, the use of chessboard calibration is a bit un-practical.
Because of that, I'm using stereoRectifyUncalibrated().
I tried to compare the results, using 2 different sets of corresponding points for the rectification:
- Points manually selected(point & click)
- Points generated from SURF and filtered with RANSAC
Input image1:
Input image2:
(Note that I do undistortion on the images before using them for rectification etc)
Rectified images with SURF and RANSAC:
(1 and 2 in that order):
Rectified images using the manually selected points(which is more inaccurate!):
Now, the thing is, looking at the result we see that the surf-version is almost perfectly rectified.(The epipolar lines are quite well aligned).
While the manually selected point version is quite badly rectified...the epipolar lines are nowhere near aligned.
But when we look at the result of openCV's sgBM() using both our rectifications:
Manual point result:
SURF point result:
The disparity/depth shown is more accurate/correct with the SURF-point(well rectified version). No surprise there.
However, the actual detected object-pixels and object-boundaries are actually a lot better on the badly rectified verison.
For instance, you can see that the pen is actually a pen and has the shape of a pen, in the bad rectified disparity map, but not in the well-rectified map.
Question is, why?
And how can I fix it?
(I tried fooling around with the sgBM() paramteres, making sure they are the same for both etc, but it does not have any effect. It is the different rectifications alone that makes the badly rectified image look good for some reason(with respect to object-boundaries)).
