I'm currently developing a multi-view stereo system but I'm confused as to how all the standard equations in Structure from Motion are affected in the presence of radial and decentering distortion. For example, the standard imaging model is that of
a) World to Camera Coordinates --> translate to camera center and rotate to match orientation of camera (R,t)
b) Perspective Projection --> x = fX/Z and y = fY/Z
c) Conversion to Pixel Coordinates --> scale by no. of pixels/m and add principal point (b and c can be combined into matrix K)
As a result, we can build a projection matrix P = K[R|t]. Thus a point X in world coordinates can be represented in pixel coordinates as u = PX.
We can also create a fundamental matrix F between two images such that a pair of corresponding point (f on image I1 and f' on image I2) satisfy the epipolar constraint;
x'Fx = 0.
The matrix F depends upon the matrix P. But, my question is that if we incorporate radial and decentering distortion, the matrix P can no longer represent the imaging model. How do we get an epipolar "curve" representation ?
Moreover, for things like triagulation back to 3D coordinates from 2D point.. how do we do it ?