Signal Processing Stack Exchange is a question and answer site for practitioners of the art and science of signal, image and video processing. It only takes a minute to sign up.
Sign up to join this community
I have been working on the topic of camera pose estimation for augmented reality and visual tracking applications for a while and I think that although there is a lot of detailed information on the task, there are still a lot of confussions and missunderstandings.
I think next questions deserve a detailed step by step answer.
It is important to understand that the only problem here is to obtain the extrinsic parameters. Camera intrinsics can be measured off-line and there are lots of applications for that purpose.
What are camera intrinsics?
Camera intrinsic parameters is usually called the camera calibration matrix, $K$. We can write
$$K = \begin{bmatrix}\alpha_u&s&u_0\\0&\alpha_v&v_0\\0&0&1\end{bmatrix}$$
where
$\alpha_u$ and $\alpha_v$ are the scale factor in the $u$ and $v$ coordinate directions, and are proportional to the focal length $f$ of the camera: $\alpha_u = k_u f$ and $\alpha_v = k_v f$. $k_u$ and $k_v$ are the number of pixels per unit distance in $u$ and $v$ directions.
$c=[u_0,v_0]^T$ is called the principal point, usually the coordinates of the image center.
$s$ is the skew, only non-zero if $u$ and $v$ are non-perpendicular.
A camera is calibrated when intrinsics are known. This can be done easily so it is not consider a goal in computer-vision, but an off-line trivial step.
What are camera extrinsics?
Camera extrinsics or External Parameters $[R|t]$ is a $3\times4$ matrix that corresponds to the euclidean transformation from a world coordinate system to the camera coordinate system. $R$ represents a $3\times3$ rotation matrix and $t$ a translation.
Computer-vision applications focus on estimating this matrix.
$$[R|t] = \begin{bmatrix} R_{11}&R_{12}&R_{13}&T_x\\R_{21}&R_{22}&R_{23}&T_y\\R_{31}&R_{32}&R_{33}&T_z \end{bmatrix}$$
How do I compute homography from a planar marker?
Homography is an homogeneaous $3\times3$ matrix that relates a 3D plane and its image projection. If we have a plane $Z=0$ the homography $H$ that maps a point $M=(X,Y,0)^T$ on to this plane and its corresponding 2D point $m$ under the projection $P=K[R|t]$ is
$$\tilde m = K \begin{bmatrix} R^1 & R^2 & R^3 & t \end{bmatrix} \begin{bmatrix} X \\ Y \\ 0 \\ 1 \end{bmatrix}$$
$$= K \begin{bmatrix}R^1&R^2&t\end{bmatrix} \begin{bmatrix} X \\ Y \\ 1 \end{bmatrix}$$
$$H = K \begin{bmatrix}R^1 & R^2 & t \end{bmatrix}$$
In order to compute homography we need point pairs world-camera. If we have a planar marker, we can process an image of it to extract features and then detect those features in the scene to obtain matches.
We just need 4 pairs to compute homography using Direct Linear Transform.
If I have homography how can I get the camera pose?
The homography $H$ and the camera pose $K[R|t]$ contain the same information and it is easy to pass from one to another. The last column of both is the translation vector. Column one $H^1$ and two $H^2$ of homography are also column one $R^1$ and two $R^2$ of camera pose matrix. It is only left column three $R^3$ of $[R|t]$, and as it has to be orthogonal it can be computed as the crossproduct of columns one and two:
$$R^3 = R^1 \otimes R^2$$
Due to redundancy it is necessary to normalize $[R|t]$ dividing by, for example, element [3,4] of the matrix.
While explaining the two-dimensional case very well, the answer proposed by Jav_Rock does not provide a valid solution for camera poses in three-dimensional space. Note that for this problem multiple possible solutions exist.
This paper provides closed formulas for decomposing the homography, but the formulas are somewhat complex.
OpenCV 3 already implements exactly this decomposition (decomposeHomographyMat). Given an homography and a correctly scaled intrinsics matrix, the function provides a set of four possible rotations and translations.
The intrinsics matrix in this case needs to be given in pixel units, that means your principal point is usually (imageWidth / 2, imageHeight / 2) and your focal length is usually focalLengthInMM / sensorWidthInMM * imageHeight.
