Camera Calibration - why do you have to move the calibration board?

Question

I understand the mathematical derivation of Zhang's Method for camera calibration but what I don't understand is why we need to move and rotate the calibration board for several frames in order to get accurate calibration results? One obvious reason is that in order to detect radial distortion you want the control points to be uniformly distributed along the field of view of the camera. However, couldn't we have also just ensured that the calibration board fills the FOV of the camera and just keep it static? Why do we have to move it around?

Thanks!

Answer 1

This is due to the optimization problem being rather high-dimensional (around 11 parameters). With only a single observation of the calibration board, there would be multiple possible combinations of parameters explaining the observed feature point locations (unless a very constrained camera-model is used). Only a sufficient number of sufficiently independent observations will properly constrain the optimization to yield a well-defined answer. Otherwise the routine will converge to some (local) minimum.

Think of this as solving a set of equations with more unknowns than equations. You will be able to find a solution, however, there would be an infinite number of solutions (one or more free parameters).

With respect to a board viewed from the front (frontoparallel orientation), filling the entire image. Now -- is this a calibration board close to the camera viewed with a wide-angle lens (short focal length), or is it a calibration board far away, viewed with a zoom lens (long focal length)? This shows that we need images with the board tilted (foreshortening).

With some constraints, you can calibrate a camera with a single image of two calibration patterns forming e.g. a 90 degree angle, or two perfectly parallel boards at differents depths.

I have written an article on the usual optimization process, which might help you understand how the process could end up not converging to the global solution: https://calib.io/blogs/knowledge-base/camera-calibration

This has little to do with the fixed grid of the imaging sensor as suggested in other answers.

Answer 2

Digital cameras sample the incoming optical EM or photonic field with a fixed grid of CCD or CMOS transistors. Since the sampling grid (rectangular or perhaps hexagonal) is not angularly symmetric, there will be different sampling artifacts depending on camera rotation relative to the image, as the alignment between sensor grid points and image grid points changes. Analysis of multiple images at different (randomized?) rotations will tend to average out those artifacts. Since a filter in front of the sensor isn't an ideal anti-aliasing filter, there will also be differences in the sampling depend on how the sampling grid micro-aligns with the incoming image field due to small linear (non-rotational) offsets as well. Again, multiple exposures will help average out these artifacts.

Answer 3

If you happen to know the distortion ratio of the image, then I think you are right. Because technically, if you can follow the mathematical derivation of camera calibration, to solve the matrix H with 11 unknows, you need at least 6 pairs of 2D undistorted image points and their corresponding 3D world coordinates. So if you know the undistorted points of like a chessboard in a image and have enough pairs, the rest of the problem is to compute H by using the least squares method to solve the overdetermined matrix.