5
$\begingroup$

I need to run comparisons on the output of hundreds of frames of simple 3D-rendered characters. Perceptually, many of the frames are exact duplicates. Based on this, I assumed something like a perceptual hash and comparison using pHash would work great. And in some cases it does...

However, I'm noticing that it doesn't tolerate minor (read: practically impossible for the human eye to see) color or contrast variations. Because of the nature of the rendered frames I received, there are some frames that are apparently different enough for pHash to conclusively decide that they are not the exact same.

Here are two example images along with a difference image:

pHash gives me a 0.473464% deviation between these images.

All those variations are enough to throw off pHash. Actually, pHash is able to tell that they are very similar, but I need a consistent low threshold to use for programmatic comparison. Other frames which actually deviate a tiny bit are sometimes considered more similar than the example above, even though they have perceptual differences. Which means I can't rely on pHash alone.

Here are two frames that are actually different (notice that his hands have moved a tiny bit):

Yet pHash gives me a difference percentage of 0.0188496%, which is less than 0.473464% from above, meaning that the frames that are perceptually different are actually considered more similar according to pHash.

I tried scaling and blurring the images a good bit to reduce any color variations, but I ended up having to do a ton of blurring to get closer. I don't want to do this too much, because I do want significant color changes to be considered as a unique frame.

For example, if there is a flash of light that illuminates the character, even if his character model is rendered in the exact same position, those frames should be considered unique:

enter image description here

Next, I tried Canny edge detection. This worked okay. Two problematic frames that are, for all intents and purposes, precise duplicates exhibited less differences:

The only issue I have with this method is that it only considers edges, which normally would be sufficient. If the character model changes at all, different edges should be detected. But this pretty much ignores significant color changes as mentioned above.

So my question is what is the best way for me to compare images like this and effectively counter insignificant color/contrast/noise variations? Seemingly pHash is ideal because of its perceptual nature, but I'm uncertain what I need to do as far as preprocessing to guarantee consistency from pHash (or some other algorithm, if it's better).

Any suggestions?

$\endgroup$
  • $\begingroup$ Can I please ask what sort of tool you are using for this? $\endgroup$ – A_A Mar 26 '15 at 9:05
  • $\begingroup$ For the image processing? ImageMagick $\endgroup$ – user15190 Mar 26 '15 at 16:21
  • $\begingroup$ immse(original,processed) in matlab $\endgroup$ – Arvind Bakshi Jul 13 '18 at 10:45
0
$\begingroup$

A simpler approach (at least to start with) would be to evaluate the absolute difference between the two images and specifically, its mean value or maybe even just the sum of it (across all pixels), and then take its histogram across your sample of pairs of images.

The histogram will follow a bimodal (or multimodal) distribution with two main modes: The lower mode corresponding to the pairs of images with slight differences and the higher mode (higher values of difference, therefore more error) to pairs of images with drastic differences. You can then use this information obtained from the histogram to work out a threshold. For more information please see: http://en.wikipedia.org/wiki/Multimodal_distribution

A few details:

  • You don't have to work on the colour images for this, you can simply convert them to greyscale. The way you do this will depend on the tools you are using.

  • The difference image that you depict in your post seems to be a bitmap, the absolute difference between two grayscale images would also "look like" a grayscale image, please see: http://homepages.inf.ed.ac.uk/rbf/HIPR2/pixsub.htm

Hope this helps, happy to amend the response if its not exactly you are looking for, I was a bit surprised to see you going for edge detection straight from the beginning so maybe I have not understood exactly the definition of "frame difference" you might be thinking about.

$\endgroup$
  • $\begingroup$ Thanks for the tips! I'll look at these. I went for edge detection because, as you can see from the differences images, the edge detection one has much less difference than the difference of the normal, full-color image. I was hoping edges were a better metric of "different content" regardless of minor color, rendering, or encoding variances throughout the image, as the first difference image illustrates. $\endgroup$ – user15190 Mar 26 '15 at 16:24
  • $\begingroup$ So I settled on an absolute error metric with a "fuzz" factor that ImageMagick can apply to suppress subtle color variations: compare -metric AE -fuzz 15% image1.png image2.png /dev/null I played with the fuzz factor until it returned 0% deviation for images that are perceptually the same and non-0% for the rest. $\endgroup$ – user15190 Mar 28 '15 at 4:23
0
$\begingroup$

Maybe you can try to calculate Minkowski distance between two images. I found that metrics useful when automatically calculate noise removal filter effectiveness.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy