This is due to the optimization problem being rather high-dimensional (around 11 parameters). With only a single observation of the calibration board, there would be multiple possible combinations of parameters explaining the observed feature point locations (unless a very constrained camera-model is used). Only a sufficient number of sufficiently independent observations will properly constrain the optimization to yield a well-defined answer. Otherwise the routine will converge to some (local) minimum.
Think of this as solving a set of equations with more unknowns than equations. You will be able to find a solution, however, there would be an infinite number of solutions (one or more free parameters).
With respect to a board viewed from the front (frontoparallel orientation), filling the entire image. Now -- is this a calibration board close to the camera viewed with a wide-angle lens (short focal length), or is it a calibration board far away, viewed with a zoom lens (long focal length)? This shows that we need images with the board tilted (foreshortening).
With some constraints, you can calibrate a camera with a single image of two calibration patterns forming e.g. a 90 degree angle, or two perfectly parallel boards at differents depths.
I have written an article on the usual optimization process, which might help you understand how the process could end up not converging to the global solution: https://calib.io/blogs/knowledge-base/camera-calibration
This has little to do with the fixed grid of the imaging sensor as suggested in other answers.