I am trying to implement feature matching algorithm between image reference and image current.

The pipeline is as follows:

  1. Gray scale Images (intensities = [0-255]) are imported.

  2. Extract ORB keypoints in both reference and current images:

Ptr<FeatureDetector> detector = ORB::create();

  1. Obtain blocks surrounding all keyPoints within specified window_size in both reference and current images. NOTE: If the blocks exceed the image upper or lower bounds, the block intensities would obviously be zero (Example: KP_A in image reference).

enter image description here

4.Instead of using existing feature matching algorithms in opencv, I am trying to utilize sum of squared of intensity differences (SSD) in the blocks acquired across ORB keypoints in reference and current images.

  1. Extracting correct features demands implementing crossCheckedMatching() to ensure features are chosen correctly.

  2. Visualize matched features.

Complete code: here.

Problem 1: My final result looks something like this: enter image description here

Whereas, I have read in few lecture notes and articles, SSD and block matching incorporated with cross checked matching method should provide decent matches. Does these matched features look any good in someone's eyes or awful after cross check matching procedure?

Problem 2: I do not know how to find the correct threshold for evaluating SSD values between two images (Empirical tuning vs. Existing logical approaches?).

Final goal, adapted from the textbook:

  1. use these matched features in RANSAC to filter outliers.

  2. Find Homography (Theory says, 4 pairs of matched points needed!)

  3. Camera Pose Estimation

  4. 3D reconstruction of environment

Apologies for such a long question, but I tried to squeeze it as much as I could.


1 Answer 1

  1. Your results look reasonable. However, you're likely to get a somewhat unreliable result: the good matches are located in a small portion of the image, while a large spread would help in getting overall more accurate estimates (but that's probably due to your images).

  2. Experimenting will work here! Otherwise, you can compute the expectation of the SSD between 2 blocks that are a rightful match under the noise level of your images.

Eventually, bad matches should be eliminated by the RANSAC part, but you may have to do a trade-off between getting many but sometimes wrong matches (loose SSD threshold) or stricter ones (at the risk of not having enough matches o estimate the pose).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.