# Projecting a 3D point into new camera coordinates

Given I have:

• a camera with estimated rigid motion 3x4 matrix $P = [R|t]$ that starts from the origin of the reference coordinate system
• an intrinsics 3x4 projection matrix $K$
• a 3D point $X$

How would I find its projection in the second camera/image? Is the following correct?

$x = K*[R|t]*X$

Your formula is correct. Here are some precisions:

• the matrix $P$ takes a scene point to an image reference coordinate frame;
• the matrix $K$ takes the coordinates of the point in the reference image frame and makes the necessary frame changes to fit with the actual camera (origin of the frame, pixel size...);
• $X$ is a 3D point in projective coordinates, i.e., $X = (x, y, z, 1)^T$, otherwise the dimensionality of the point is not correct with the projection matrix;
• $x$ will be actually a column of 3 rows: $x = (u,v,w)^T$, and the final image coordinates are given by diving $u$ and $v$ by $w$.

• the equation assumes an ideal pinhole camera model. Thus, the coordinates are undistorted. You can actually find all these equations by yourself using basic triangular geometry and Thales's theorem;
• you don't add the projective coordinate ("the 1") twice! The product $P \times (X,1)^T$ will create as output a projected (2D) point that has 3 coordinates, i.e., a 2D point in projective coordinates. What you need only is to "remove" the projective coordinate by dividing by $w$ at the end;
• to get the order always right, the best way is to think of these equations as I wrote then in plain English. You take a 3D world and project it onto the world plane given by the camera position ($P$, extrinsic parameters). Then, you consider what happens for this particular camera model/sensor (intrinsic parameters). The extrinsic parameter matrix describes the camera with respect to the world, while the intrinsic parameters are always teh same wherever you put your camera.
• I was missing the part of dividing the coordinates by $w$! Thanks! Also: is $x$ the point in the distorted image (think pinhole camera model)? Jun 5, 2014 at 19:25
• And the order multiplication is first $L = P * X$, then $x = K * L$? In this case I should add the fictitious coordinate $1$ twice right? Jun 5, 2014 at 19:38
• I've updated the answer. Jun 6, 2014 at 7:29
• So, according to your last point, $K$ would be an affine 3x3 and not a 3x4 like I wrote in the question? Otherwise the matrix math wouldn't add. And the last column of $P$ would always be $[0,0,0]^{T}$? Jun 6, 2014 at 7:40
• Also, when you divide by $w$, are you basically projecting onto the camera plane $z=1$? Jun 6, 2014 at 7:43

As I can't comment yet, I have to put these comments into a separated answer

• If you are working in Homogeneous Coordinates then $P$ is 4x4 (as it takes 3D Points in and out and these ones are described by 4x1 Vector in HC)
• Furthermore you don't have to confuse
• the “Camera Pose Matrix” hence the 4x4 $[R_{c}|t_{c}]$ matrix describing Camera Pose with respect to some world related Reference Frame, with the
• the “Extrinsic Camera Matrix” hence the 4x4 $P=[R|t]$ matrix which transforms 3D Points from the World Reference to the Camera Reference Frame

To be precise there are other 2 Reference Frames

• the Sensor Reference Frame, which is again measured in meters but it is after the Projection and the
• the Image Reference, which is measured in pixels so it is after applying $u0, v0$ and the pixel quantization

So to perform the Projection you need the Extrinsic Camera Matrix $P$ not the Camera Pose Matrix $[R_{c}|t_{c}]$, however they are strongly related as one is the inverse of the other

$$P = [R_{c}|t_{c}]^{-1} = [R_{c}^{t}|-R_{c}^{t}t_{c}]$$

• That comes from the fact in RT Convention first the Rotation and then the Translation is applied
• So the $R_{c}$ inverse is $R_{c}^{t}$ by the properties of the Rotation Matrix and $t_{c}$ inverse is obviously $-t_{c}$
• If you apply the Translation after the Rotation you get the above mentioned result

• Finally the $K$ Intrinsic Camera Matrix is 3x4 in HC (as the Projection removes one dimension)