# Is SSIM a measure of mutual information?

As i have seen contradictory information online: is the quality measure SSIM between two images (one base-line, one distorted) directly correlated to the mutual information between the two. One one hand, MSSIM can be partially interpreted as normalized mean square errors between the two on the other hand there are papers online that explicitely extend MSSIM by a mutual information term, suggesting they are not the same. I know this is not very in-depth, but for now i'm look for a simple yes/no -answer so I can decide in which direction to extend my further research. Thanks!

• wait, you want to research, but you want a simple yes/no answer? You know that the formula for mutual info is different for the one of SSIM, so there's your "no" answer that you could have gotten yourself; but: you actually seem to be rather interested in this topic, and seem capable enough, so I think this warrants that you ask more specifically about what you really care about. – Marcus Müller Jun 8 '18 at 12:46
• Yeah the formulas are different but as SSIM is based on MSEs and standard deviations of the the images and MSE indirectly maximizing mutual information when being minimized, I would like to know if there is a underlying correlation. I'm doing my own research but was unable to find a definite answer. – Jane Dough Jun 8 '18 at 12:55
• well, there's an underlying correlation for most image types I can think of, but you can devise data with no to little mutual information but high SSIM, and vice versa. – Marcus Müller Jun 8 '18 at 13:01
• (That statement – mutual info and SSIM are correlated for most images, is pretty intuitive, the other stems from looking at the formulas intensely; so again, I don't think I'm telling you anything new! Please, explain why you are asking this.) – Marcus Müller Jun 8 '18 at 13:03
• So as i understand you could theoretically have two different images (with different probability distributions) that have a high SSIM value? Maximizing SSIM does not nescessarily maximize mutual information? – Jane Dough Jun 8 '18 at 13:43

## 2 Answers

...is the quality measure SSIM between two images (one base-line, one distorted) directly correlated to the mutual information between the two [?]

The short answer to this is "yes" but it tells you nothing. Because the question is a little bit ill posed.

You have two signals, let them be signals or images, it doesn't matter, you have two observations. And you want to assess how much they are alike.

What does alike mean? You might have an idea about this already, if it is sound do they sound similar? If it is images, do they look similar?

So, things look alike when they co-vary...Sound pressure over time covaries between two sound clips, brightness (reflected light flux) over space covaries between two images.

One of the first metrics of assessing how much alike two signals (images) are that someone might encounter is cross correlation .

If you employ (normalised) cross correlation, on your baseline image (i.e calculate its autocorrelation), you will observe the highest value at the centre of the spatial cross correlation. A value of 1.0.

Now, notice here, we decoupled the concept of co-variance from the way by which we assess it. Co-variance is this notion that things might be going up and down at the same time/place. Cross correlation is a quantification of that notion.

And, there are others. Cross correlation, mutual information, transfer entropy, granger causality, partial correlation, ... and many others.

ALL of these are correlated with each other. Their ranges, sign and even interpretation might be different, but if you adjust for all of these, in general, all of these metrics go in one direction when the observations co-vary and in another when they don't co-vary.

Let's turn to the SSIM, the SSIM is the product of three factors: luminance, contrast and structure with the following expressions: (Image captured on wikipedia)

Where the $\mu$'s denote means and the $\sigma$'s denote variances with the $\sigma_{xy}$ in particular denoting covariance. By the way, you might want to notice the similarity of the $l,c,s$ components with the covariance and even the correlation coefficient.

Luminance doesn't even count, if I remove the mean from two images, the luminance component goes to zero. It is a little bit more difficult to zero the Structure but I can do it, if the two images do not covary with each other, a condition that is met if the two images are pure white noise. This will leave the contrast with some value that would have to be equal to the separate variances.

Long story short: SSIM contains a covariance term, covariance and mutual information are quantifications of the same concept, therefore, SSIM and mutual information are correlated.

BUT, this tells you nothing. The point is how are these two correlated and the interpretation. So, there is a different mapping between cross correlation and mutual information because mutual information has that $\log$ factor in it and, if it is mutual information then would it assume memory of some length?

So now we are on the next level, WHAT are you trying to achieve? To then move on to WHICH aspect of your signal do you need to quantify (and how) in order to get there?

Hope this helps.

• Sorry, didn't check back for a while. Great answer, thanks a lot! – Jane Dough Jun 26 '18 at 18:25
• @JaneDough Thanks, glad you found it helpful. All the best. – A_A Jun 26 '18 at 22:49

SSIM cares about perceptual likeness, based on model of human visual system. Mutual Information as a concept has no such built-in bias.