Image Segmentation based on coordinates from reference points

Question

I have a set of colors in a card, as below (I can modify the card as required)

I have 4 reference points at four corners. I have a bunch of colors at known coordinates (in mm) from these reference points.

A user clicks a picture of this card on their phone (Android) (assume good lighting condition)

I can calibrate the distance from the number of pixels between the identified reference patterns and the known actual distance (in mm) between them

How do I extract each of these color values from the picture on the Android phone locally?

What I thought so far:

1. Detect the reference patterns and get their coordinates.

2. I know the actual distance in mm

3. From the coordinates and actual distance, get pixels-per-mm

4. I know the actual distance in mm of each of these colors

5. Offset by that distance (pixels) for each of the colors and find the average color in a small radius around that point.

How do I achieve this?

Answer 1

You can do something much simpler.

With a single MATLAB line to calculate the mask:

mI = im2double(imread('https://i.sstatic.net/Nf5oe.jpg'));
mM = all((mI == 0) | (mI == 1), 3);

You will get:

What you need to do next:

• Do some cleaning and identify the columns and rows.
• Reconstruct the missing squares in columns / rows.
• Calculate the average / median color per square.