Given I have:

  • a camera with estimated rigid motion 3x4 matrix $P = [R|t]$ that starts from the origin of the reference coordinate system
  • an intrinsics 3x4 projection matrix $K$
  • a 3D point $X$

How would I find its projection in the second camera/image? Is the following correct?

$x = K*[R|t]*X$


Your formula is correct. Here are some precisions:

  • the matrix $P$ takes a scene point to an image reference coordinate frame;
  • the matrix $K$ takes the coordinates of the point in the reference image frame and makes the necessary frame changes to fit with the actual camera (origin of the frame, pixel size...);
  • $X$ is a 3D point in projective coordinates, i.e., $X = (x, y, z, 1)^T$, otherwise the dimensionality of the point is not correct with the projection matrix;
  • $x$ will be actually a column of 3 rows: $x = (u,v,w)^T$, and the final image coordinates are given by diving $u$ and $v$ by $w$.

A few answers to your comments:

  • the equation assumes an ideal pinhole camera model. Thus, the coordinates are undistorted. You can actually find all these equations by yourself using basic triangular geometry and Thales's theorem;
  • you don't add the projective coordinate ("the 1") twice! The product $P \times (X,1)^T$ will create as output a projected (2D) point that has 3 coordinates, i.e., a 2D point in projective coordinates. What you need only is to "remove" the projective coordinate by dividing by $w$ at the end;
  • to get the order always right, the best way is to think of these equations as I wrote then in plain English. You take a 3D world and project it onto the world plane given by the camera position ($P$, extrinsic parameters). Then, you consider what happens for this particular camera model/sensor (intrinsic parameters). The extrinsic parameter matrix describes the camera with respect to the world, while the intrinsic parameters are always teh same wherever you put your camera.
| improve this answer | |
  • $\begingroup$ I was missing the part of dividing the coordinates by $w$! Thanks! Also: is $x$ the point in the distorted image (think pinhole camera model)? $\endgroup$ – aledalgrande Jun 5 '14 at 19:25
  • $\begingroup$ And the order multiplication is first $L = P * X$, then $x = K * L$? In this case I should add the fictitious coordinate $1$ twice right? $\endgroup$ – aledalgrande Jun 5 '14 at 19:38
  • $\begingroup$ I've updated the answer. $\endgroup$ – sansuiso Jun 6 '14 at 7:29
  • $\begingroup$ So, according to your last point, $K$ would be an affine 3x3 and not a 3x4 like I wrote in the question? Otherwise the matrix math wouldn't add. And the last column of $P$ would always be $[0,0,0]^{T}$? $\endgroup$ – aledalgrande Jun 6 '14 at 7:40
  • $\begingroup$ Also, when you divide by $w$, are you basically projecting onto the camera plane $z=1$? $\endgroup$ – aledalgrande Jun 6 '14 at 7:43

As I can't comment yet, I have to put these comments into a separated answer

  • If you are working in Homogeneous Coordinates then $P$ is 4x4 (as it takes 3D Points in and out and these ones are described by 4x1 Vector in HC)
  • Furthermore you don't have to confuse
    • the “Camera Pose Matrix” hence the 4x4 $[R_{c}|t_{c}]$ matrix describing Camera Pose with respect to some world related Reference Frame, with the
    • the “Extrinsic Camera Matrix” hence the 4x4 $P=[R|t]$ matrix which transforms 3D Points from the World Reference to the Camera Reference Frame

To be precise there are other 2 Reference Frames

  • the Sensor Reference Frame, which is again measured in meters but it is after the Projection and the
  • the Image Reference, which is measured in pixels so it is after applying $u0, v0$ and the pixel quantization

So to perform the Projection you need the Extrinsic Camera Matrix $P$ not the Camera Pose Matrix $[R_{c}|t_{c}]$, however they are strongly related as one is the inverse of the other

$$ P = [R_{c}|t_{c}]^{-1} = [R_{c}^{t}|-R_{c}^{t}t_{c}] $$

  • That comes from the fact in RT Convention first the Rotation and then the Translation is applied
  • So the $R_{c}$ inverse is $R_{c}^{t}$ by the properties of the Rotation Matrix and $t_{c}$ inverse is obviously $-t_{c}$
  • If you apply the Translation after the Rotation you get the above mentioned result

  • Finally the $K$ Intrinsic Camera Matrix is 3x4 in HC (as the Projection removes one dimension)

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.