0
$\begingroup$

Currently I have a fairly accurate (50 µm) and large dataset from a 3D scanner and a set of high resolution pictures. Sadly the 3D scanner (a Nikon/Metris arm-based CMM that was hanging around the lab) doesn't record colour data, but this would be exactly what I need to extract the actual features I'm interested in. I started out by correcting the lens distortion for the camera, and also accounted for the non-linear distortion. What prevents me from doing this manually in software like Meshlab or Geomagic is that I have an extremely large set of objects to scan, which makes it highly unrealistic to do it by hand.

I took the liberty to add a series of reference points which are visible to both the 3D scanner and conventional camera. I am able to detect these features on both the point cloud and images automatically. Sadly I must admit I underestimated a big part of this task: I now wish to automatically align the image versus the point cloud based on these reference points and project the image onto the point cloud or mesh. The reference points from the point cloud are in a XYZ format, while those from the image are 2D coordinates in the plane. They are already matched, so that presumably - according to a fellow Ph.D. student - difficult part is already done. And while I can think of a few ways to align, distort, and project the images this would most likely be a large investment of time from my part. I was wondering if any pre-existing libraries for this exist or at least if there are papers which present a method to do so. Based on the fact that software like Geomagic is able to do it I'd like to avoid reinventing the wheel. However I must humbly admit I can't seem to find them as I am probably using the wrong terminology.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
0
$\begingroup$

If you're only left with matching 3D points to their corresponding 2D parts, this is a fairly task and called PnP pose estimation. OpenCV has a lot of options to solve this problem. Here is one tutorial, which you could maybe benefit.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you, didn't know they named it pose. (I was looking for position.) I was curious if you have any idea how critical the focal distance would be on a distortion corrected image? I would imagine if it's in the correct range (a few mm off) that it'll be able to resolve it correctly as the only thing that changes would be the distance to the object? $\endgroup$
    – Bart Plovie
    Commented Mar 14, 2016 at 9:29
0
$\begingroup$

"Project the image onto the point cloud or mesh": this not a trivial step either, depending on your application requirements (e.g., do you need large contiguous textures?), the result quality you expect (e.g. image edge blending, relighting), and whether you want to perform some further 3D-related manipulation on the images (e.g 3D painting).

Suggest to look for papers on projective texture mapping. Or, once you have the poses of all cameras, you could just import the data in a 3D editing application (Blender, Maya, 3DStudioMax, ...) and go from there.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The difficult part in this case is actually the distortion correction, which I already did. Though I wrote an algorithm that is able to correct for it based on the entered reference points, but I haven't found the need to use it yet for this particular case. The surfaces are relatively small, and the camera is of sufficiently high resolution that all features can be extracted form a single shot. Additionally it's a test and measurement application so it doesn't have to look nice. I can also tune it compared to a CT scanner dataset, so that helps. $\endgroup$
    – Bart Plovie
    Commented Mar 18, 2016 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.