0
$\begingroup$

For a project I'm doing, I'm trying to model a scene based on a phone camera's video input, and then insert arrows into it that indicate direction in three dimensions.

I've looked around, and this paper looks quite interesting, and I suppose I could try to adapt its methods: http://kevinkarsch.com/publications/sa11-lowres.pdf

However, it has the problem that users must input some reasonable spatial constraints into it beforehand, whereas doing this for every frame of a live and constantly changing video is unreasonable.

Another thing that I found allows reconstruction of a 3D object from a few photos of it in various directions, but I can't seem to find the original source anymore.

Any ideas?

Improve this question
$\endgroup$
  • $\begingroup$ note that phones often have gyroscope/compass/accelerometer which can give you good information about which way the camera is pointing, too. bitbucket.org/apacha/sensor-fusion-demo $\endgroup$ – endolith Nov 14 '16 at 15:30
3
$\begingroup$

SLAM(Simultaneous Localization and Mapping) algorithms can be used to for 3D reconstruction. They offer solutions for both monocular as well as stereo cameras. With single camera they estimate depth with few images and reconstruct the scene. You can find some of the open source solutions here.

Real time 3D reconstruction can done using ORBSLAM and it is also opensource.Here is the related paper for monocular SLAM.

Improve this answer
$\endgroup$
  • $\begingroup$ Oh I've heard of SLAM!! I think that was the reconstruction thing I had forgotten. ORBSLAM looks awesome though. $\endgroup$ – oink Nov 14 '16 at 20:54
2
$\begingroup$

Let us first assume you can produce estimates of the camera state (position and attitude) via sensors, a filter like a Kalman Filter, and a (simple) model for the camera itself. Using this information, you could then match features between a sequence of images and use those matches and camera state estimates to estimate relative 3D coordinates of those features in space.

Using these features and their locations in 3D, you could then use interpolation schemes to build textured 3D models based on the sequences of images.

Obviously one of the tough points with this is noisiness in state estimates for the camera each frame/image in the sequence. But as you match features between multiple images, noise can be smoothed out and allow you to produce more stable results.

Improve this answer
$\endgroup$
  • $\begingroup$ Yeah, there's certainly enough input information to reconstruct the scene with good accuracy (phone sensors are actually amazing), but the matching-features process itself seems to be the hard part. $\endgroup$ – oink Nov 14 '16 at 20:58
  • $\begingroup$ @cchan3141 Yeah it can be tough to find what feature matching approach is a good balance of robustness and efficiency for a given problem. Not to mention the ability to make good matches is dependent on using a sufficiently detailed descriptor for your problem. $\endgroup$ – spektr Nov 14 '16 at 22:49
0
$\begingroup$

Since you are having a sequence of images from a scene or an object on your phone, I suggest to use a fairly simple method called Structure from Motion1.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.