# 3D reconstruction of moving object

I am pretty new to the field of 3D reconstruction and overwhelmed by the large amounts of different methods. I am trying to find the correct method for my use case:

Setting: I have a stationary camera. In front of this camera, an object is moving from left to right in a straight line. The camera only sees a part of the object as it is positioned near the object.

Goal: I don't need the complete 3D model of the object. I just would like to estimate the shape of the part the camera is seeing. Here: The curve of the middle of the ball.

Data: I have following information:

• Frames per second the camera is taking pictures.
• Rough distance from camera to object (this metric isn't constant and can slightly change).
• Multiple keypoints on the object: I have a segmentation model using Deep Learning that segments parts of the object. From those segments I can obtain keypoints (e.g. top right corner of segmentation mask)

Possible Solutions: I thought of following approaches:

• As I have the segmentations of different parts on the object I was thinking about using a voxel carving method (shape from silhouettes): but the angles are limited as the object moves in a straight line parallel to the camera.

• The segmentation masks allow me to extract keypoints. As I also have the number of frames the camera takes pictures I was thinking that the problem may be solved using multiple view algorithms, but I don't know how to extract the camera position from the frame rates.

• Welcome. I think this is a crucial problem. There might be additional information to help: what can you say about the settings (global shape of the object, is it smooth or not, lighting/shadow, ratio of distance to diameter, types of features you are interested in) Commented May 10, 2020 at 14:37
• Basically I can assume that I already have perfectly matching keypoints. Also, which is new, I now how many mm's the object moved. Can I use something like the 8 point algorithm or won't this work, because the object moves in a horizontal line? Commented May 11, 2020 at 6:30