Minimization of Essential matrix

Question

A problem in computer vision and 3d reconstruction is getting the camera's intrinsics parameters. A common solution is to use an object in which one knows the measurements of the shape before hand, such as a checker board. The issue with this method is that it must be done every time the camera parameter's change such as focal length and magnification.

I'm trying to implement the camera self-calibration discussed in A Simple Technique for Self-Calibration. The essential matrix is constrained by its two singular values. This can be used to recover the camera's intrinsics without doing a manual calibration (i.e., with a checkerboard). I am a little confused by how the cost function can be minimized. Here's what I understand so far:

essential matrix $$E=K_2^TFK_1$$

intrinsic matrix $$K=\begin{bmatrix}\alpha_x & s & u_0 \\ 0 & \epsilon\alpha_x & v_0 \\ 0 & 0 & 1\end{bmatrix}$$

• $\alpha_x$ product of focal length and magnification factor [solve]
• $\epsilon$ aspect ratio [assume provided, I guess from camera or EXIF data?]
• $u_0 v_0$ are the coordinates of the principal point [assume 0, 0]
• $s$ skew [assume 0]

cost function $$C(K_i,i=1..n)=\sum_{ij}^n(\sigma1_{ij}-\sigma2_{ij})/\sigma1_{ij}$$ the $\sigma$s are the singular values of $K_j^TF_{ij}K_j$

Question: How is this cost function being minimized?

Answer 1

I guess this is a straightforward non-linear optimization problem (to be solved with Newton variations, such as Trust-Region methods), where you don't even need to compute the Jacobian analytically. It appears to me that the optimization problem is written over $K_i$, and thus is the input to the cost function. To compute the cost, at each call to this function, you basically compute the singular values of $K_i$ and compute the cost according to equation $4$ (in the paper). As your input parameter is $K$, the derivatives are computed over the elements of $K$. That makes your optimization $4$ dimensional (or $5$ if you consider the skew) per camera. The derivatives are computed automatically, and you don't need to care about that. If you are using MATLAB,lsqnonlin would work for you.

The weight computation is explained in a detailed fashion in the paper, so I skip this part.

Having checked the paper once again, I have noticed that the authors are actually using the numerical differentiation scheme I have mentioned. If you would like to understand more deeply how to differentiate an SVD, you might like to check out this one or this.