For a project I'm doing, I'm trying to model a scene based on a phone camera's video input, and then insert arrows into it that indicate direction in three dimensions.
I've looked around, and the paper Rendering synthetic objects into legacy photographs (The presentation in SIGGRAPH Asia 2011) looks quite interesting, and I suppose I could try to adapt its methods.
However, it has the problem that users must input some reasonable spatial constraints into it beforehand, whereas doing this for every frame of a live and constantly changing video is unreasonable.
Another thing that I found allows reconstruction of a 3D object from a few photos of it in various directions, but I can't seem to find the original source anymore.
Any ideas?