How to Downsample The Chrominance Components

Question

So I am doing an image compression project for college and am trying to create a visual demonstration of how downsampling chrominance components can reduce the amount of digital information while being hardly noticeable to the average person. While doing research for this project, I came across this video where the guy gives a demonstration of downsampling that leads to there being 100x less color in the image he uses as an example. I was hoping to do something similar but am unsure as to how he was able to downsample the chrominance components in an image by a factor of 10x in each direction.

Looking at the standard ratios for chroma subsampling(/downsampling) the best I could find seems to be 4:2:0 which would downsample the chrominance by a factor of 2x in each direction. It's possible that might be enough for a demonstration, but I was hoping I might could push it a little further.

As a side question, does anyone have any recommendations as to how best to apply the downsampling color to my raw image? I have some ideas involving either photoshop or GIMP, but I was wondering if anybody had a better suggestion.

Answer 1

For demonstration purposes GIMP is just fine. It can split an image to three separate YUV/YCbCr images. You can then manually resize the UV/CbCr images by any amount to downsample the chroma. Then upsample back to original resolution and recombine the image again.

Answer 2

You seem to be confused between the difference between what you want to do, the standards that exist for doing it with video, and tools that you might use to do it with -- apparently -- still images.

What you want to do

You want to separate out the chrominance channel, then you want to average it in 10x10 blocks (for a factor of 100), then you want to make 10x10 blocks all of the same chrominance, then you want to mix the gray scale back in. There's lots of ways to do this. Note that chrominance probably adds oddly -- it may be better to set the luminance (Y) channel to a constant gray, then subsample the resulting image, then put the "real" luminance back in.

Video Standards

These are designed to respond to market forces, and aim to deliver not-too-crappy video to the consumer. So yes, the amount of chrominance sub-sampling is going to be limited. Choosing video standards as your tool for doing this is probably not wise.

Tools to use

The answer on using Gimp (or Photoshop, if you swing that way) looks good to me. If I were going to do this on a movie, as opposed to a single still picture, I'd separate the thing into frames using ffmpg, do the subsampling using imagemagik, then put it all back together with ffmpg -- and before you ask, no, I don't know the details of how to do this; I'm just 99.44% certain that it's possible, and that I could figure it out.

Answer 3

JPEG compression relies on a number of techniques while reducing an image's storage size. Primarily it's the DCT stage which accounts for the gross bit reduction. This stage is controlled by the quality parameter. However, color is also used to advantage as follows.

It's experimentally verified that our eyes are more sensitive to brightness resolution than the chroma (color) resolution. To take advantage of this charactersitics, raw RGB image data is first converted into a YUV (or digital counterpart YCbCr) color space, in which Y represents grayscale brightness information and U-V (Cb-Cr) components represent the chroma; color hue and color saturation informations. Then you can perform a reduction in the U-V component signal bandwidths by lowpass filtering them (which equates to a reduction in effective color spatial resolution) and then see that the resulting reconstruction of the image looks almost the same.

Note that this is not the approach taken in image-video compression codecs. As I have said, they reduce the information based on a 2D-DCT transformation, which will result in a substantially better image quality and more data reduction. Keep in mind however that what's performed in the DCT domain (on 8x8 image blocks) is somehow equivalent to a variable cutoff lowpass filtering on the spatial domain. But it will be very effective in achieving bit reduction while keeping the quality high, unlike what's demonstrated below as a complete lowpass filtering of the whole channels at once.

The following MATLAB code demonstrates the effect.

I = double( imread('C:\matlab\...\Vegatable256.bmp') ); % read the Vegatable image

R = I(:,:,1);   % Red channel
G = I(:,:,2);   % Green channel
B = I(:,:,3);   % Blue channel


% S1: JFIF COLOR-SPACE CONVERSION RGB-YCbCr MATRIX:
% ------------------------------------------------
% Y'CbCr (601) from "digital 8-bit R'G'B'  all signals in {0, 1, 2, ..., 255}
M = [0.299    ,  0.587     , 0.114 ; 
    -0.168736 , -0.331254  , 0.5   ;
     0.5      , -0.418688  ,-0.081312; ];

% inverse matrix is used to convert YCbCr into RGB
Mi = inv(M); % use with (Cb-128, Cr-128) to get RGB

% S2 - Convert RGB to YCbCr :
% ---------------------------
Y  = 0.299*R + 0.587*G + 0.114*B;
Cb = 128  - 0.168736*R - 0.331264*G + 0.5*B;
Cr = 128  + 0.5*R - 0.418688 *G- 0.081312*B;

% S3 - Design a simple 2D lowpass filter :
b1 = fir1(32,0.1);
b2 = b1'*b1;            

% S4 - Apply lowpass filtering on the Y,Cb,Cr channles.
Y2 = conv2(Y,b2,'same');
Cb2 = conv2(Cb-128,b2,'same');
Cr2 = conv2(Cr-128,b2,'same');


% S5 - Reconstruct various coombinations :
% ----------------------------------------
% 1 - I1: only the color Cb, Cr is lowpass filtered
I1 = zeros(S(1),S(2),3);
I1(:,:,1) = Mi(1,1)*Y + Mi(1,2)*Cb2 + Mi(1,3)*Cr2; % R;
I1(:,:,2) = Mi(2,1)*Y + Mi(2,2)*Cb2 + Mi(2,3)*Cr2; % G;
I1(:,:,3) = Mi(3,1)*Y + Mi(3,2)*Cb2 + Mi(3,3)*Cr2; % B;
I1 (I1 > 255) = 255;
I1( I1 < 0) = 0;

% 2 - Only Luminance Y is lowpass filtered 
I2 = zeros(S(1),S(2),3);
I2(:,:,1) = Mi(1,1)*Y2 + Mi(1,2)*(Cb-128) + Mi(1,3)*(Cr-128); % R;
I2(:,:,2) = Mi(2,1)*Y2 + Mi(2,2)*(Cb-128) + Mi(2,3)*(Cr-128); % G;
I2(:,:,3) = Mi(3,1)*Y2 + Mi(3,2)*(Cb-128) + Mi(3,3)*(Cr-128); % B;
I2 (I2 > 255) = 255;
I2( I2 < 0) = 0;

% 3- All of then lowpass filtered
I3 = zeros(S(1),S(2),3);
I3(:,:,1) = Mi(1,1)*Y2 + Mi(1,2)*Cb2 + Mi(1,3)*Cr2; % R;
I3(:,:,2) = Mi(2,1)*Y2 + Mi(2,2)*Cb2 + Mi(2,3)*Cr2; % G;
I3(:,:,3) = Mi(3,1)*Y2 + Mi(3,2)*Cb2 + Mi(3,3)*Cr2; % B;
I3 (I3 > 255) = 255;
I3( I3 < 0) = 0;


% DISPLAY THEM:
%I4 = zeros(2*S(1),2*S(2));
I4 = [I , I1 ; I2 , I3];
figure,imshow(I4/255);
title('UL: original, UR: Chroma only LPF to 0.1 bandwidth');
xlabel('LL: Y only LPF, LR: All channels LPF');  

The output will be :

As you can see, when you reduced the chroma bandwidth to one tenth (in both directions, resulting in 1/100 overall) of the default, (in the upper right corner), there is little image quality loss. Sharpness and colors are retained. Note that originally Cb and Cr channels were lowpass type signals; i.e., they contained little energy in the high frequency regions which is thrown out by the filter.

However when you lowpass filter the Brightness (Luminance or Y) channel with the same filter, the result is a dramatic loss in image quality as shown on the lower right corner. This is because the brightness channel has significant high energy content and our eyes are sensitive to it.

Answer 4

This demonstrates till what level of Cb, Cr downsampling human eye can't detect:

from PIL import Image
import numpy as np

def downsample_cb_cr(ycbcr, ds_level):
    w, h = ycbcr.size
    ycbcr_np = np.asarray(ycbcr)
    ycbcr_res = ycbcr.resize((w//ds_level, h//ds_level))
    ycbcr_ds = ycbcr_res.resize((w, h))
    ycbcr_ds_np = np.array(ycbcr_ds)
    ycbcr_ds_np[:,:,0] = ycbcr_np[:,:,0]
    return Image.fromarray(ycbcr_ds_np, mode='YCbCr')

def main():
    img = Image.open("images/lena.jpeg")
    img_np = np.array(img)
    ycbcr = img.convert('YCbCr')
    for level in [2,5,10,20,30,40,50,80,100,200]:
        ycbcr_new = downsample_cb_cr(ycbcr, level)
        ycbcr_new.save("images/lena_chroma_ds_level_{}.jpeg".format(level))

if __name__=='__main__':
    main()