2
$\begingroup$

I recently find myself an interesting problem to solve.

Basically I got 2 partially overlapped point clouds (in real world) that I want automate the registration process. My naive approach currently is to render the 2 point cloud from top down, colourised by intensity and convert to grey-scale images.

I will then use SIFT to find the features and descriptors of the features, and try to match them from one point cloud to the other. As expected, copy and paste the code from OpenCV tutorial is not so successful. I would love some guidance on this and possibly another approach for auto registration of 2 point cloud?

Point cloud 1 from top view: enter image description here

Point cloud 2 from top view: enter image description here

A side by side in comparison and roughly where it should match: enter image description here

How the features actually match up: enter image description here

And the brilliant code I used:

import numpy as np
import cv2 as cv
import matplotlib.pyplot as plt
from time import time

start_time = time()

img1 = cv.imread("1.png")
img2 = cv.imread("2.png")

gray1 = cv.cvtColor(img1, cv.COLOR_BGR2GRAY)
gray2 = cv.cvtColor(img2, cv.COLOR_BGR2GRAY)
print(f"Load 2 images - Elapsed time: {time() - start_time}")

start_time = time()
sift = cv.SIFT_create(nOctaveLayers=1, contrastThreshold=0.05, edgeThreshold=15, sigma=1.6)
kp1, des1 = sift.detectAndCompute(gray1, None)
kp2, des2 = sift.detectAndCompute(gray2, None)
print(f"SIFT - Elapsed time: {time() - start_time}")

img3=cv.drawKeypoints(gray1,kp1,img1,flags=cv.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
img4=cv.drawKeypoints(gray2,kp2,img2,flags=cv.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
f, ax = plt.subplots(1,2) 
ax[0].imshow(img3, cmap='gray')
ax[1].imshow(img4, cmap='gray')
plt.show()

start_time = time()
bf = cv.BFMatcher()
matches = bf.knnMatch(des1, des2, k=2)
print(f"Match - Elapsed time: {time() - start_time}")

# Apply ratio test
good = []
for m, n in matches:
    if m.distance < 0.75 * n.distance:
        good.append([m])
# cv.drawMatchesKnn expects list of lists as matches.
img3 = cv.drawMatchesKnn(gray1, kp1, gray2, kp2, good, None, flags=cv.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS)
plt.imshow(img3)
plt.show()
$\endgroup$
1
  • $\begingroup$ had you find any progress on this ? Ive got the same issue at work and couldnt find a lot of guidance.. bassiclly, ive got 2 clouds and try to produce a new one with the no overlapping area of each one in addition to only one of the original clouds. Thanks. $\endgroup$
    – JuanAn
    Oct 27, 2022 at 19:41

1 Answer 1

1
$\begingroup$

Hope this helps because this is what I am going to do, because I have the exact same problem. If you look at the results, there is a way to do a 3 point matching between the images.

Define a matching parameter, then check the points, you can particle swarm optimization checking the matching parameter and hope for the best.

That was the best solution for us, and with a low performance result, it will save us like 3 hours of work per day. Which is not bad if I can say so myself.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.