Slicing an Image into Tiles According to Content

Question

Given an image that contains an NxM grid of icons with text below them, somewhat like this:

I'm wondering how to split it up into the individual icons with text. Some wrinkles:

• The image won't always be the same size (nor will the icons); it might show 18 icons as 6x3, or 16 icons as 4x4, etc.
• The window might be scrolled, such that the top and/or bottom row of icons are "chopped off" in the image (I can keep or discard the incomplete ones, whatever's easiest).
• The background behind the icons isn't white. It's a complicated pattern with gradients that aren't purely horizontal or vertical, and where there are some discontinuities—so it's not as easy as "find a horizontal line where all the pixels are the same color value" or "where each color value is consistently +x from the previous one". But the icons are in very vibrant colors, and the background is pretty subdued; I'm hoping that might be detectable?

Essentially what I need is a generalized way to detect the "space" between the rows and columns of icons, and know what position to "cut" right in the middle of that space, like I was using a paper cutter.

I have to automate this by programming, so I'm less interested in an already-existing software tool that can do it, and more interested in how to approach the problem (what strategies/algorithms would help solve it). I know so little about this area that I'm not even sure what this problem is called. :) Thanks!

Answer 1

Thanks to user Ash for giving me a key insight: Measuring the HSL (hue, saturation, and lightness) of the pixels can help detect where the gaps are. If I take one horizontal row of pixels that goes straight through my icons (in this example, 5 icons per row), and graph the saturation and lightness, you can see how they become smooth within the vertical gaps:

And if I take one horizontal row of pixels that happens to be in a horizontal gap, you can see the saturation and lightness only reflects the gradient of the background, so this isn't a good place to measure:

By detecting where the S/L starts to vary wildly, and where it smooths back out, that gives me the left and right edges of each icon. Then I can do the same for the vertical dimension.

Answer 2

Under the following assumptions (Written for the vertical lines, same for the horizontal):

1. The background is uniform.
2. There at least a gap of 2 background pixels between the closest objects on different columns.
3. There is a gap of at least 2 pixels of background at the left and at the right.

The I'd solve it would be:

1. Convert the image into grayscale.
2. Calculate the variance along the columns.
3. Find areas where at least 2 adjacent values are zero (Numerically).
4. Set the column divider to be the middle of the areas above.

This is how it looks:

At the top image one could see the input image and the vertical dividers calculated by the algorithm (In red).

At the middle image we can see the the thresholded variance. The variance is calculated per column and the graph shows when the variance is larger than eps() or not. When the value is 0 it means there were practically no variance -> Background.

At the last image we can see the segments. For segments which are of value 0 the middle was chosen to be a divider.

In order to get the horizontal dividers one could apply the same algorithm on the rows (Or transpose the image).

The code is available at my StackExchange Codes Signal Processing Q82921 GitHub Repository (Look at the SignalProcessing\Q82921 folder).