Take the 2-minute tour ×
Signal Processing Stack Exchange is a question and answer site for practitioners of the art and science of signal, image and video processing. It's 100% free, no registration required.

I have been working on the topic of camera pose estimation for augmented reality and visual tracking applications for a while and I think that although there is a lot of detailed information on the task, there are still a lot of confussions and missunderstandings.

I think next questions deserve a detailed step by step answer.

  • What are camera intrinsics?
  • What are camera extrinsics?
  • How do I compute homography from a planar marker?
  • If I have homography how can I get the camera pose?
share|improve this question
    
I'm fuzzy on the renormalization you do: 1. H is the homography found from the data using some procedure (say SVD). 2. inv(K)*H=A is the thing you work with here. Then you make q1 = a1/norm(a1) and q2 = a2/norm(a2) as orthonormal columns of a rotation matrix, and make q3=q1xq2... Then you take t/(something) to get the translation vector. How is it you can just divide q1 and q2 by possibly different things, and how do you pick what to divide t by? Or is the idea that the SVD procedure and multiplication by inv(K) give something close but not quite orthogonal/orthonormal rotation matrix, so th –  user2600616 Aug 17 '13 at 3:53

1 Answer 1

It is important to understand that the only problem here is to obtain the extrinsic parameters. Camera intrinsics can be measured off-line and there are lots of applications for that purpose.

What are camera intrinsics?

Camera intrinsic parameters is usually called the camera calibration matrix, $K$. We can write

$$K = \begin{bmatrix}\alpha_u&s&u_0\\0&\alpha_v&v_0\\0&0&1\end{bmatrix}$$

where

  • $\alpha_u$ and $\alpha_v$ are the scale factor in the $u$ and $v$ coordinate directions, and are proportional to the focal length $f$ of the camera: $\alpha_u = k_u f$ and $\alpha_v = k_v f$. $k_u$ and $k_v$ are the number of pixels per unit distance in $u$ and $v$ directions.

  • $c=[u_0,v_0]^T$ is called the principal point, usually the coordinates of the image center.

  • $s$ is the skew, only non-zero if $u$ and $v$ are non-perpendicular.

A camera is calibrated when intrinsics are known. This can be done easily so it is not consider a goal in computer-vision, but an off-line trivial step.

What are camera extrinsics?

Camera extrinsics or External Parameters $[R|t]$ is a $3\times4$ matrix that corresponds to the euclidean transformation from a world coordinate system to the camera coordinate system. $R$ represents a $3\times3$ rotation matrix and $t$ a translation.

Computer-vision applications focus on estimating this matrix.

$$[R|t] = \begin{bmatrix} R_{11}&R_{12}&R_{13}&T_x\\R_{21}&R_{22}&R_{23}&T_y\\R_{31}&R_{32}&R_{33}&T_z \end{bmatrix}$$

How do I compute homography from a planar marker?

Homography is an homogeneaous $3\times3$ matrix that relates a 3D plane and its image projection. If we have a plane $Z=0$ the homography $H$ that maps a point $M=(X,Y,0)^T$ on to this plane and its corresponding 2D point $m$ under the projection $P=K[R|t]$ is

$$\tilde m = K \begin{bmatrix} R^1 & R^2 & R^3 & t \end{bmatrix} \begin{bmatrix} X \\ Y \\ 0 \\ 1 \end{bmatrix}$$

$$= K \begin{bmatrix}R^1&R^2&t\end{bmatrix} \begin{bmatrix} X \\ Y \\ 1 \end{bmatrix}$$

$$H = K \begin{bmatrix}R^1 & R^2 & t \end{bmatrix}$$

In order to compute homography we need point pairs world-camera. If we have a planar marker, we can process an image of it to extract features and then detect those features in the scene to obtain matches.

We just need 4 pairs to compute homography using Direct Linear Transform.

If I have homography how can I get the camera pose?

The homography $H$ and the camera pose $K[R|t]$ contain the same information and it is easy to pass from one to another. The last column of both is the translation vector. Column one $H^1$ and two $H^2$ of homography are also column one $R^1$ and two $R^2$ of camera pose matrix. It is only left column three $R^3$ of $[R|t]$, and as it has to be orthogonal it can be computed as the crossproduct of columns one and two:

$$R^3 = R^1 \otimes R^2$$

Due to redundancy it is necessary to normalize $[R|t]$ dividing by, for example, element [3,4] of the matrix.

share|improve this answer
2  
I think it is misleading to say that calibration is "easy and not the goal of CV". In usual case we also need to estimate the distortion parameters. Instead of self calibration I would recommend planar calibration (Zhang - A Flexible New Technique for Camera Calibration) as it is more flexible if separated calibration procedure can be done. You also have small error in "If I have homography how can I get the camera pose?" as you don't take into the account the calibration (H_{calib} = K^-1H). –  buq2 Jul 1 '12 at 5:34
1  
camera pose from homography is wrong. There are several way to do it' some of them are highly non-trivial. –  mirror2image Jul 5 '12 at 11:52
    
I don'r see why it is wrong. I compute it this way and works. Why do you say it is wrong? –  Jav_Rock Jul 5 '12 at 12:29
2  
You wrote in the last section that H^1 and R^1 and equal, but in the 3rd section you state that H=K[R T] which would mean that R^1 is actually K^-1H^1. But this is not strictly true as there is infinite number of H which will satisfy the equations and will cause problems when solving R^1, R^2 and T (the unknown scale). Your answer disregards robust intrinsic and distortion calibration and some of the equations are wrong for which reason this is not a good answer for the question. –  buq2 Jul 5 '12 at 22:23
    
Yes, I was missing the kalibration matrix in step three as I took this from my code and I multiply by K in a different function of the codes. –  Jav_Rock Jul 6 '12 at 7:58

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.