# Should I consider rgb values of a pixel as one value?

I was reading this paper for a project work.

We train on 1.6 million 32*32 color images that have been preprocessed by subtracting from each pixel its mean value over all images and then dividing by the standard deviation of all pixels over all images.

I've trouble distinguishing between "from each pixel its mean value over all images" and "standard deviation of all pixels over all images".

Since, I'm dealing with color images, can I just take rgb values of each pixel as one value or should I calculate the mean and SD for every color differently?

For example if I have r=255, g=255, b=255, can I take pixel value as (in binary), (r<<16)+(g<<8)+b ?

I believe that you will want to process each channel separately over all of the images. Otherwise a mean and variance will have very little meaning. And if you convert to grayscale before training, then the network can only retrieve grayscale images, which doesn't seem to match with what the paper describes.

I've trouble distinguishing between "from each pixel its mean value over all images" and "standard deviation of all pixels over all images".

This processing will requires two passes:

On The first pass, you will compute the mean pixel values of each channel, and the variance over the entire set of pixels in a channel. When you are finished, you should have 3075 values: one mean value per pixel per channel (32*32*3=3072), and one variance per channel (3).

On the second pass you will modify the images by taking the subtracting from each pixel the mean you found in the first pass, and dividing by the standard deviation from the first pass.

Let $\mu_r,i,j$ be the mean value of the red channel at the $i,j$-th pixel, and let $\sigma_r^2$ be the variance of the red channel over all pixels in all images. Then the new value corresponding to the red channel of the $i,j$-th pixel in the $k$-th image $r_{i,j,k}$ becomes

$$\tilde{r}_{i,j,k} = \left(r_{i,j,k} - \mu_{r,i,j}\right)/\sqrt{\sigma_r^2}$$

Where

$$\mu_{r,i,j} = \frac{1}{K}\sum_k r_{i,j,k}$$ $$\sigma_r^2 = \frac{1}{IJK-1}\sum_{i,j,k} r_{i,j,k}^2 - \frac{1}{IJK-1}\sum_{i,j,k} r_{i,j,k}$$

This is my interpretation of the paper. However, since the focus of the paper is on autoencoders and deep belief networks, the image preprocessing step is only mentioned in passing. Like most academic papers, it gives enough details for the reader to understand what was done without necessarily giving enough details to reproduce the result without lots of reading between the lines.

Good luck to you!

• Thanks.. For my sake, can you expand sigma-r part? Over all pixels over all images, means calculating mean over all pixels linearly over all images and use it in calculating SD? – pinkpanther Oct 13 '13 at 9:43
• Sure. I have made an edit to my answer, let me know if it helps. – nispio Oct 14 '13 at 17:57
• Why would there need to be a mean value per pixel per channel. Why not just a mean per channel over all images? – CMCDragonkai Feb 8 '19 at 6:40

For example if I have r=255, g=255, b=255, can I take pixel value as (in binary), (r<<16)+(g<<8)+b ?

While this number would identify the colour, it would not help with a mean or standard deviation. e.g. pure blue would be 255 but pure red would be 16711680

The paper is unclear so you have a few choices:

1. Process each channel individually
2. Convert the image to grayscale, find the mean and std dev, and use this for each channel.

There may be other methods that use different colour spaces.

• +1 Thanks for your time.... but paper says 3072 visible units 3072=32*32*3, it seems 3 channels are taken. It seems 3 channels are taken individually. What do you think? – pinkpanther Oct 12 '13 at 15:05