Minimization of Essential matrix

A problem in computer vision and 3d reconstruction is getting the camera's intrinsics parameters. A common solution is to use an object in which one knows the measurements of the shape before hand, such as a checker board. The issue with this method is that it must be done every time the camera parameter's change such as focal length and magnification.

I'm trying to implement the camera self-calibration discussed in A Simple Technique for Self-Calibration. The essential matrix is constrained by its two singular values. This can be used to recover the camera's intrinsics without doing a manual calibration (i.e., with a checkerboard). I am a little confused by how the cost function can be minimized. Here's what I understand so far:

essential matrix $$E=K_2^TFK_1$$

intrinsic matrix $$K=\begin{bmatrix}\alpha_x & s & u_0 \\ 0 & \epsilon\alpha_x & v_0 \\ 0 & 0 & 1\end{bmatrix}$$

• $\alpha_x$ product of focal length and magnification factor [solve]
• $\epsilon$ aspect ratio [assume provided, I guess from camera or EXIF data?]
• $u_0 v_0$ are the coordinates of the principal point [assume 0, 0]
• $s$ skew [assume 0]

cost function $$C(K_i,i=1..n)=\sum_{ij}^n(\sigma1_{ij}-\sigma2_{ij})/\sigma1_{ij}$$ the $\sigma$s are the singular values of $K_j^TF_{ij}K_j$

Question: How is this cost function being minimized?

• Good question. This might be worth asking over at math.SE, as this seems like a pretty pure math problem if you can distill the application-specific details out. – Jason R Dec 5 '11 at 14:55
• Thanks, I was originally trying to decide between the two sites. I've discovered some new things that I can use in separate questions. – Athena Dec 6 '11 at 20:21

I guess this is a straightforward non-linear optimization problem (to be solved with Newton variations, such as Trust-Region methods), where you don't even need to compute the Jacobian analytically. It appears to me that the optimization problem is written over $K_i$, and thus is the input to the cost function. To compute the cost, at each call to this function, you basically compute the singular values of $K_i$ and compute the cost according to equation $4$ (in the paper). As your input parameter is $K$, the derivatives are computed over the elements of $K$. That makes your optimization $4$ dimensional (or $5$ if you consider the skew) per camera. The derivatives are computed automatically, and you don't need to care about that. If you are using MATLAB,lsqnonlin would work for you.