# Image registration / fusion optimal similarity metrics

I have a theoretical question about optimal similarity metrics for comparing data sets.

In reading this linked paper, pp. 488-489 [1], I read the following 2 interesting statements. On page 488:

In the ... scenario in which $A$ and $B$ differ only by Gaussian noise, then it can be shown that $\rm SSD$ [Sum of Squares Difference] is the optimum measure

And on page 489:

If the intensities in images $A$ and $B$ are linearly related, then the correlation coefficient $\rm CC$ can be shown to be the ideal similarity measure

Are there are other conditions under which other similarity metrics [e.g. Mutual Information] can be proven to be optimal, or under which the above 2 metrics ($\rm SSD$ and $\rm CC$) are also optimal? The linked PDF focuses on image registration. I am thinking mostly in terms of image registration/image fusion similarity metrics, but would also be interested in more general similarity between any 2 or more datasets (don't have to be images).

[1] P. A. Viola, Alignment by maximization of mutual information. Ph.D. thesis, Massachusetts Institute of Technology, 1995.

• @sambajestson - What about euclidean distance? Commented Mar 23, 2015 at 16:59

I'm not sure what is meant here by the "optimal" similarity metric for a given situation. The textbook you linked to references the following thesis:

[87] P. A. Viola, Alignment by maximization of mutual information. Ph.D. thesis, Massachusetts Institute of Technology, 1995.

...but I can't find it anywhere online after a quick search.

From my point of view:

• Sum of square differences measures the difference in pixel intensities, so it is useful when pixel intensities are very similar in both images
• Normalized cross-correlation measures linear correlation between pixel intensities, so it is useful when image intensities are linearly correlated
• Mutual information is a measure of general statistical dependence between pixel intensities, so it is useful when image intensities have a non-linear statistical dependence

If SSD works for a given pair of images, I don't see why NCC or MI wouldn't do the job just as well, since they are more general. They may be slower since the metric takes longer to compute, but I don't understand why one would be more "optimal" than the other.

• – Emre
Commented Apr 22, 2015 at 17:33
• @Emre Thanks. There's a famous paper with the same name so I had a hard time tracking down the thesis it's based on. Commented Apr 22, 2015 at 19:04