`SSIM(YUV)` vs. `SSIM(Y)`: Values Are Very Different

Question

I'm seeing some weird results when trying to compare image qualities in a few different color spaces. Have a look at my graph:

I want to compare with another publication, which reported PSNR(YUV) and SSIM(Y). However, I don't understand how I can have SSIM(YUV)=0.9 for the highest quality point on the right, but only have SSIM(Y)=0.7 for the exact same image. How can there be such a big difference? Is this normal? Or am I doing something wrong?

For completeness, here are reference image and the image under test:

And the MATLAB code I use for converting RGB to YUV:

function YUV=rgb2yuv(RGB)
R = RGB(:,:,1);
G = RGB(:,:,2);
B = RGB(:,:,3);

Y = 0.299   * R + 0.587   * G + 0.114 * B;
U = 0.5 - 0.168736 * R - 0.331264 * G + 0.5 * B;
V = 0.5 + 0.5 * R - 0.418688 * G - 0.081312 * B;
%   ^^^ I tried to use 0.0 as well, but SSIM values are identical


YUV=cat(3,Y,U,V);
end

Answer 1

I figured it out: the ssim() function in MATLAB, does not handle RGB images, but instead threats it as a 3D-image (with the third dimension time for example, to measure the quality of a video). So it interpreted my 3-color-component image as a video with 3 frames.