# How to Apply the SSIM Measure on RGB Images?

I want to compare the similarity of two images by looking at its structure and colors. I need a measure that takes both structural and color fidelity into account. When I checked the formulation of original paper, it is said only x and y mean and variance is measured. Does SSIM account for structure only or does it take color similarity into account too?

What usually can be done is calculate it per channel and have the average (Or weighted average).

So if you have $${R}_{1}, {G}_{1}, {B}_{1}$$ and $${R}_{2}, {G}_{2}, {B}_{2}$$ you would do:

$${SSIM}_{r} \left( {R}_{1}, {R}_{2} \right), {SSIM}_{g} \left( {G}_{1}, {G}_{2} \right), {SSIM}_{b} \left( {B}_{1}, {B}_{2} \right)$$

Then $${SSIM}_{rgb} = {w}_{r} {SSIM}_{r} + {w}_{g} {SSIM}_{g} + {w}_{b} {SSIM}_{b}$$.

For instance, MATLAB's ssim() does the above with $${w}_{r} = {w}_{g} = {w}_{b} = \frac{1}{3}$$.

SciKit Image's compare_ssim() also apply it channel wise using the multichannel flag: • I just made a 512*512 all orange jpeg and the same size yellow jpeg file. Then I measured the similarity by using SSIM and result is 0.60. Then I made a red one and the similarity between orange and red is 0.8. It compares colors too I suppose. At first when I checked the name, because there is "Structural", it only measured structure of the images but my little experiment shows that it measures colors too. Do you confirm this?
– Lyrk
May 16 at 8:53
• Do you really need to separate the image into its RGB components to get the exact similarity? Can't I use the original image and the image to be measured directly without separating into 3 components?
– Lyrk
May 16 at 9:11
• By design it only applied to a single channel image. What do you use to apply the comparison? If you use MATLAB, what it does is exactly as above with ${w}_{r} = {w}_{g} = {w}_{b} = \frac{1}{3}$.
– Royi
May 16 at 9:12
• You don't need to separate as in 3 different arrays, just implement it in such way.
– Royi
May 16 at 9:13
• I use Scikit image in Python. github.com/scikit-image/scikit-image/blob/main/skimage/metrics/… There is a "multichannel" argument there. If I use it I hope I do not have to measure channels separately.
– Lyrk
May 16 at 9:24