I decided to post this answer here because a while back, this came up as the top result in Google and its suggestions helped me. So I decided to share my experience too.
Having spent countless hours trying to get the best stereo calibration on a Kinect, I shared my tips and findings in a blog post here.
Although it is geared towards stereo calibration and more specifically Kinect, I believe the tips will help anyone who is trying to calibrate a camera.
Also, in case I should die someday or forget to renew my hosting, here is a modified quote from the post:
- Make sure you have the largest possible calibration pattern. Follow what I said above. Get a nice pattern professionally printed.
Make sure each square is at least 8cm x 8cm. Also, make sure one side
of the calibration pattern has an odd number of squares and the other
side has an even number of squares (e.g 9×6 or 7×8). It’s important
for detecting the pose of the target correctly. Also, some toolboxes
will not be able to detect the pattern if this requirement is not met.
As mentioned before, the patterns I used which are suitable for
printing on large sheets are uploaded here (for 9cm squares) and here
(for 10cm squares).
- Make sure your printed pattern has enough white border around it, otherwise it may not be easily detected by most toolboxes.
- Make sure the Kinect does not move. I mounted in on a tripod.
- Try to get as many images of the calibration target as you can. My best calibration was obtained using 300 images, at distances as low as
0.5 meter to as far as 10 meters from the camera. Make sure you rotate the pattern around X, Y and Z axes. Also try to “tile” the view with
images taken at the same distance: i.e take one image, move the target
to the next tile in the field of view, take another one and repeat
until you’ve “tiled” all of the current field of view. The goal is to
cover the entire field of view at each distance as much as possible.
- Use MATLAB’s stereo calibration app if possible. It allows you to get rid of the outliers after each calibration phase.
It's almost 3 years after all this. A couple of years of industry experience has taught me something new: if your task heavily relies on the correctness of camera calibration, consider building a quick UI that allows you to manually fine-tune extrinsic/intrinsics and see immediate visual results. This way, you can focus on other parts of your system while not having to think about how bad camera calibration is affecting you. Once everything is ready, you can decide on the best method to calibrate your cameras with.