7
$\begingroup$

Given an image that contains an NxM grid of icons with text below them, somewhat like this:

file browser grid of icons with text

I'm wondering how to split it up into the individual icons with text. Some wrinkles:

  • The image won't always be the same size (nor will the icons); it might show 18 icons as 6x3, or 16 icons as 4x4, etc.
  • The window might be scrolled, such that the top and/or bottom row of icons are "chopped off" in the image (I can keep or discard the incomplete ones, whatever's easiest).
  • The background behind the icons isn't white. It's a complicated pattern with gradients that aren't purely horizontal or vertical, and where there are some discontinuities—so it's not as easy as "find a horizontal line where all the pixels are the same color value" or "where each color value is consistently +x from the previous one". But the icons are in very vibrant colors, and the background is pretty subdued; I'm hoping that might be detectable?

Essentially what I need is a generalized way to detect the "space" between the rows and columns of icons, and know what position to "cut" right in the middle of that space, like I was using a paper cutter.

I have to automate this by programming, so I'm less interested in an already-existing software tool that can do it, and more interested in how to approach the problem (what strategies/algorithms would help solve it). I know so little about this area that I'm not even sure what this problem is called. :) Thanks!

$\endgroup$
2
  • 1
    $\begingroup$ I think your last bullet has good insight into a solution. You need to find some distinguishable feature between rows/cols that contain icons, and those that don't. As a first pass, I'd look at the variance of color in the lines. Background variance should be relatively low and lines containing icons should be high. And by variance of color, I don't necessarily mean RGB. Consider HSV, HSL, or a combination of the three. See en.wikipedia.org/wiki/HSL_and_HSV#Color_conversion_formulae. Then you need to group lines containing text and icons together probably based on width. $\endgroup$
    – Ash
    May 6 at 21:36
  • $\begingroup$ Nice one. Could you share 2-3 images as examples? I think I have an idea. $\endgroup$
    – Royi
    May 6 at 22:01

2 Answers 2

5
$\begingroup$

Under the following assumptions (Written for the vertical lines, same for the horizontal):

  1. The background is uniform.
  2. There at least a gap of 2 background pixels between the closest objects on different columns.
  3. There is a gap of at least 2 pixels of background at the left and at the right.

The I'd solve it would be:

  1. Convert the image into grayscale.
  2. Calculate the variance along the columns.
  3. Find areas where at least 2 adjacent values are zero (Numerically).
  4. Set the column divider to be the middle of the areas above.

This is how it looks:

enter image description here

At the top image one could see the input image and the vertical dividers calculated by the algorithm (In red).

At the middle image we can see the the thresholded variance. The variance is calculated per column and the graph shows when the variance is larger than eps() or not. When the value is 0 it means there were practically no variance -> Background.

At the last image we can see the segments. For segments which are of value 0 the middle was chosen to be a divider.

In order to get the horizontal dividers one could apply the same algorithm on the rows (Or transpose the image).

The code is available at my StackExchange Codes Signal Processing Q82921 GitHub Repository (Look at the SignalProcessing\Q82921 folder).

$\endgroup$
1
$\begingroup$

Thanks to user Ash for giving me a key insight: Measuring the HSL (hue, saturation, and lightness) of the pixels can help detect where the gaps are. If I take one horizontal row of pixels that goes straight through my icons (in this example, 5 icons per row), and graph the saturation and lightness, you can see how they become smooth within the vertical gaps:

enter image description here

And if I take one horizontal row of pixels that happens to be in a horizontal gap, you can see the saturation and lightness only reflects the gradient of the background, so this isn't a good place to measure:

enter image description here

By detecting where the S/L starts to vary wildly, and where it smooths back out, that gives me the left and right edges of each icon. Then I can do the same for the vertical dimension.

$\endgroup$
1
  • $\begingroup$ I added an answer with a MATLAB code. Really simple and yet nice question :-). $\endgroup$
    – Royi
    May 22 at 5:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.