3
$\begingroup$

I am trying to find an algorithm to determine the similarity between 2 screenshots of the same screen of a mobile application. I have tried structural similarity index, Peak signal-to-noise ratio and other similar IS algorithms. These algorithms generate a lot of false positives. I have also tried open open cv2's matchTemplate, ORB, SIFT and FLANN matching which do not seem to produce promising results. I feel the solution seems to be right around the corner but I am still not able find the one that produces a high success rate. Any help that would guide me towards a better solution is appreciated.

Example Images:

Screenshot 1

enter image description here

UPDATE:

Here's my use case: Users upload screenshots as a proof of entering a particular screen of an app. The system must be able to detect whether the screenshot actually contains the screen of the app.

$\endgroup$
4
  • 1
    $\begingroup$ I don't know the answer to your question (alas -- it's a good one), but I'm sure it'll he helpful if you edit your question to say how you want it to be similar. In ascending order of difficulty, consider, first, two images that have a few details different but are otherwise the same image at different scales, second, your two images but with different fonts, spacing, and backgrounds, and third (and quite a jump), Picasso's "Nude Descending a Staircase" and DaVinci's "Mona Lisa". Each of these pairs of things are similar, but less and less to a machine as you go down the list. $\endgroup$
    – TimWescott
    Nov 25, 2022 at 16:23
  • $\begingroup$ I have edited the question to add the exact use case I am trying to implement. Hope if helps. $\endgroup$
    – Vinoth RJ
    Nov 25, 2022 at 18:25
  • $\begingroup$ One difficulty in comparing the two images is that they are different sizes and scale: i.stack.imgur.com/SLWcc.png $\endgroup$
    – Rob
    Nov 26, 2022 at 6:52
  • 2
    $\begingroup$ Can you embed a QR code on that screen of the app? You should always look for simple solutions first… :) Otherwise you’d have to compare color schemes (if constant), and then read the text items and match them to expected text items. A pixel to pixel comparison is never going to work here. $\endgroup$ Nov 27, 2022 at 16:24

1 Answer 1

0
$\begingroup$

You better solve it at the system engineering level by entering some kind of a simple identifier.

Specifically on the images you shared, I'd probably work in 2 directions:

  1. Identify the specific icons.
    You may create different template for different screen resolutions.
    There not many different resolution in the market of iOS and Android, so basic template matching will work.
  2. Use OCR to identify the text and compare word / sentences to what you expect.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.