I need to do a pixel-for-pixel comparison between a 'candidate' image generated by a deep-neural-net against an 'ideal' image. Rather than just comparing RGB values, I'd like to do a comparison that is bounded by limitations in human visual perception; i.e., if a human can't tell (or almost can't tell) the difference between the two images, then the difference returned would be zero (or almost zero).


Put another way, before comparing my two RGB images, I'd like to convert them into some color-space with the following characteristics:

  • Adjacent colors in the (discreet) target space are just above the difference threshold for human perception.

  • The same is true for luminance.

Obviously, this conversion would be lossy. But, two images that are perceptually equivalent in the RGB space would map to the same pattern of ones and zeros when converted into this target space.

If this target space was constructed such that any non-linear features of human visual perception were (more or less) linear in the target space, then it ought to be much simpler to establish a perceptual difference threshold using only a single real value.


The human vision limitations I remember are something along the following lines:

  • Color ranges for the three different types of cone cells
    (i.e. low/high frequency limits)

  • Color resolution
    (i.e. minimum distinguishable difference between colors)

  • Luminance resolution
    (i.e. minimum distinguishable difference between brightness levels)

  • Frequency limitations for one or both?
    (e.g. at what frequency does alternating black and white become indistinguishable from gray, or yellow and blue from green)?

I know that JFIF's lossy compression algorithm was based partly on this kind of perceptual information.

I searched for articles, but kept coming up with stuff like the PerceptualImageDiff (PID) project on SourceForge/GitHub. PID might do what I need but it definitely does a whole lot more I don't need. It's also big and slow, and it's in C++. My images are tiny, so performance in a Python implementation should be fine. I'm leaving PID as a my last option, because teasing out just the relevant algorithms/formulas I need may be more work than just starting from theory/formulas/algorithms in some online resource.

Can anyone point me to a good resource for theory/algorithms/formulas pertaining to this perceptual stuff?

  • $\begingroup$ I think that the question is rather broad the way it is posed. In general, this falls under the topic of "Perceptual quality modelling" and there is a huge amount of work both in sound as well as imaging. Your problem seems to be a special case of one of the existing models but it does not seem to be sufficiently formulated yet. Which of these parameters are you more interested in? Is your resolution high enough? $\endgroup$ – A_A Jan 24 '18 at 7:12
  • $\begingroup$ Leaving 'frequency' out of it, it's a bit simpler. The idea would be to transform both images into some new discreet color/luminance mapping where each step in either luma or chroma would be the minimum that a human could discern. Then, a pixel-by-pixel comparison between the two would be immune to differences that a human couldn't perceive, and Euclidean distance would be more or less linear wrt perceptual difference. $\endgroup$ – Scott Smith Jan 24 '18 at 21:43
  • $\begingroup$ This is looking promising - "Just Noticeable Distance" (JND): github.com/scikit-image/scikit-image/blob/master/skimage/color/… $\endgroup$ – Scott Smith Jan 24 '18 at 23:19