Camera calibration using checkerboard instead of 3D calibration object

Question

What is the purpose and advantage of using a checkerboard than a 3D calibration object for camera calibration ?

With a 3D calibration object (e.g. a transparent cuboid like a table), I can take one image with a camera, then find the corners of the cuboid, which gives me 8 2D-3D correspondences.

Since a projective camera matrix $P$ has 11 DoF, I have enough equations to obtain $P$.

Now with a checkboard given one image, I obtain the internal camera matrix and external kamera matrix.

However, the external camera matrix describes the change of basis from the checkerboard coordinate system to the camera coordinate system, and thus changes from image to image. Thus is the idea of the checkerboard to take multiple images to obtain a more accurate estimation of the internal camera parameters, which should be same across all taken images ?

So for image i, we have 2D points $p_{i}$ with corresponding 3D points $P_{i}$ (on the checkerboard). Let $K_{i}$ be the internal camera matrix, $E_{i}$ be the external camera matrix and $M = \begin{pmatrix}1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0\end{pmatrix}$ the projective camera model.

So I am assuming that the optimization problem that is being solved is

$$\min_{K,E_1,\ldots,E_m} \sum_{j=1}^{m}\sum_{i=1}^{n} || (K M E_{j}) P_{i}- p_{i,j}||^2$$

so that we obtain one internal camera matrix that fits to all images ?

Accordingly, am I right that it is necessary to do both, the calibration with the checkerboard to get an accurate estimate of $K$ and then the calibration with the 3D calibration object to obtain the extrinsic camera matrix, so that the 3D calibration object defines the world coordinate system ?

Answer 1

Now with a checkboard given one image, I obtain the internal camera matrix and external kamera matrix.

No, you don't. You get a single coordinate transformation matrix that contains only the ratio between focal length and object distance. You cannot split that into an internal camera matrix (focal length) and external matrix (object distance) without further information.

If you want that extra information from only a single image, then the points of your calibration target must not be all in a single plane. This is why people use cuboid calibration targets. Alternatively, you can take at least two images of checkerboards from different perspectives, which also allows you split focal length and object distance.

Accordingly, am I right that it is necessary to do both, the calibration with the checkerboard to get an accurate estimate of K and then the calibration with the 3D calibration object to obtain the extrinsic camera matrix, so that the 3D calibration object defines the world coordinate system?

No, this is not necessary. A single image of a cuboid calibration target is enough to determine both the internal and external camera matrix and a single image of a checkerboard gets you nothing.