2
$\begingroup$

I am pretty new to the field of 3D reconstruction and overwhelmed by the large amounts of different methods. I am trying to find the correct method for my use case:

Setting: I have a stationary camera. In front of this camera, an object is moving from left to right in a straight line. The camera only sees a part of the object as it is positioned near the object.

enter image description here

Goal: I don't need the complete 3D model of the object. I just would like to estimate the shape of the part the camera is seeing. Here: The curve of the middle of the ball.

Data: I have following information:

  • Frames per second the camera is taking pictures.
  • Rough distance from camera to object (this metric isn't constant and can slightly change).
  • Multiple keypoints on the object: I have a segmentation model using Deep Learning that segments parts of the object. From those segments I can obtain keypoints (e.g. top right corner of segmentation mask)

Possible Solutions: I thought of following approaches:

  • As I have the segmentations of different parts on the object I was thinking about using a voxel carving method (shape from silhouettes): but the angles are limited as the object moves in a straight line parallel to the camera.

  • The segmentation masks allow me to extract keypoints. As I also have the number of frames the camera takes pictures I was thinking that the problem may be solved using multiple view algorithms, but I don't know how to extract the camera position from the frame rates.

Improve this question
$\endgroup$
2
  • $\begingroup$ Welcome. I think this is a crucial problem. There might be additional information to help: what can you say about the settings (global shape of the object, is it smooth or not, lighting/shadow, ratio of distance to diameter, types of features you are interested in) $\endgroup$
    – Laurent Duval
    Commented May 10, 2020 at 14:37
  • $\begingroup$ Basically I can assume that I already have perfectly matching keypoints. Also, which is new, I now how many mm's the object moved. Can I use something like the 8 point algorithm or won't this work, because the object moves in a horizontal line? $\endgroup$
    – oezguensi
    Commented May 11, 2020 at 6:30

1 Answer 1

Reset to default
2
$\begingroup$

This problem is also known as "4D reconstruction" or "spatio-temporal reconstruction." Several methods have recently been developed to solve this problem, such as "occupancy flow."

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.