I want to find the orientation of 2x2 checkerboard cubes, like these:
That is to say, I want to recover their positions and rotations within the image relative to the camera.
In my particular case, I care more about the rotation than the position (and the depth barely matters at all). I know the colors ahead of time, and can change them to be more convenient. I want to pick up the cubes, so a hand may be in the frame. The lighting may not be uniform, but the background should be simple (e.g. carpet, table, pavement).
I do have a partially working solution, which I've included below as an answer (it makes the question too specific to include it here). But I'm looking for suggestions on how to approach the problem and what techniques to apply.
Other things I've been considering:
- Use a line detection algorithm to detect edges, and use those to seed hypotheses (instead of the center saddle point).
- Do a cartesian-to-polar transform at the detected saddle points to turn the angled lines into straight lines (expensive?).
- Score hypotheses by using them to unskew a face then dot product-ing that against a reference image of the face (but what about lightning changes?).
- Random combinations of dilation and erosion throughout (invariably doesn't seem to help).