I've been working for a while on 3D building reconstruction from old photos (c. 1904). In all of these cases, the cameras and buildings are long gone, making standard camera calibration impossible. To even get an affine reconstruction, the only thing I have to rely on (that I know of) is vanishing points.
Even ignoring image measurement error, vanishing points have a few problems:
- I'm at the mercy of the craftspeople who built the buildings. In many cases, the allegedly parallel lines don't converge very well to a single point. And the actual orthogonality of the vanishing points is unknown.
- I'm at the mercy of the photographer. In most cases, the photos are nearly 2-point perspectives. There is usually enough tilt that I can estimate the third vanishing point, but it's a poor estimate, and it's often a couple of orders of magnitude further away than the other two, which I assume further degrades the estimate.
Initially, I had hoped that I could get a rough reconstruction and improve it by manually "tweaking" the vanishing points. Unfortunately, there's no clear relationship between the reconstruction error and the vanishing point error. I've looked some at bundle adjustment, but in most cases I only have a few photos that include any particular feature.
Does anyone have any suggestions of how to improve or supplement the vanishing point accuracy? Here are a couple of things I've tried:
- Calculate the trifocal tensor from three images, and use it to check the accuracy of a vanishing point by transferring the same vanishing point from the other two images.
- Manually correct the reconstruction, find the (3D) homography between it and the original reconstruction, and use this to correct the cameras.
Do these seem reasonable? Does anyone have any other/better suggestions?