# Projecting a 3D point onto Multiple Cameras in the Same Projective Frame

I need a sanity check. I think I understand the following, but I just can't get it to work.

Suppose that I calculate a trifocal tensor $$T$$ from three sets of corresponding points in three images (call the sets $$x$$, $$x'$$, and $$x''$$). From $$T$$, I calculate three cameras, $$P$$, $$P'$$, $$P''$$, using Algorithm 15.1 in Hartley & Zisserman's Multiple View Geometry in Computer Vision (Algorithm 14.1 in this PDF copy), so that all three cameras are in the same projective frame.

Now I triangulate two points $$x_i$$ and $$x'_i$$ from the first two sets, resulting in a 3D point $$X_i$$. As expected, $$PX_i = x_i$$ and $$P'X_i = x'_i$$, up to a scale factor.

Here's the question: since $$P''$$ is in the same projective frame, if I project $$X_i$$ with $$P''$$, (i.e., $$P'' X_i$$), shouldn't I get $$x''_i$$ (up to a scale factor)?

It seems like it should work (otherwise, what good is it for $$P''$$ to be in the same projective frame?), but I've been over my code dozens and dozens of times, and I can't find the bug. I just need to know if my understanding is correct, or if I'm missing something fundamental.

## 1 Answer

It appears that my problem was normalization. The mentioned Algorithm for calculating the cameras makes use of the epipoles $$e'$$ and $$e''$$ (i.e., the images of the first camera's focal point in the second and third cameras).

Since I normally think of the epipoles as (homogeneous) image points, I was normalizing them so that the third coordinate was 1. Apparently, they need to be normalized to unit length, or else the third camera doesn't come out right.

The formula for the third camera is $$P'' = [(e’’e’’^T – I)[T^T_1, T^T_2, T^T_3]e’ | e’’]$$ where $$T$$ is the trifocal tensor. I suspect that only the term $$(e''e''^T - I)$$ requires $$e''$$ to be unit-length, since the others will just contribute to an overall scale factor. But I'm no mathematician.