I am pretty new to the field of 3D reconstruction and overwhelmed by the large amounts of different methods. I am trying to find the correct method for my use case:

Setting: I have a stationary camera. In front of this camera, an object is moving from left to right in a straight line. The camera only sees a part of the object as it is positioned near the object.

enter image description here

Goal: I don't need the complete 3D model of the object. I just would like to estimate the shape of the part the camera is seeing. Here: The curve of the middle of the ball.

Data: I have following information:

  • Frames per second the camera is taking pictures.
  • Rough distance from camera to object (this metric isn't constant and can slightly change).
  • Multiple keypoints on the object: I have a segmentation model using Deep Learning that segments parts of the object. From those segments I can obtain keypoints (e.g. top right corner of segmentation mask)

Possible Solutions: I thought of following approaches:

  • As I have the segmentations of different parts on the object I was thinking about using a voxel carving method (shape from silhouettes): but the angles are limited as the object moves in a straight line parallel to the camera.

  • The segmentation masks allow me to extract keypoints. As I also have the number of frames the camera takes pictures I was thinking that the problem may be solved using multiple view algorithms, but I don't know how to extract the camera position from the frame rates.

  • $\begingroup$ Welcome. I think this is a crucial problem. There might be additional information to help: what can you say about the settings (global shape of the object, is it smooth or not, lighting/shadow, ratio of distance to diameter, types of features you are interested in) $\endgroup$ – Laurent Duval May 10 at 14:37
  • $\begingroup$ Basically I can assume that I already have perfectly matching keypoints. Also, which is new, I now how many mm's the object moved. Can I use something like the 8 point algorithm or won't this work, because the object moves in a horizontal line? $\endgroup$ – oezguensi May 11 at 6:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.