0
$\begingroup$

I was reading this paper for a project work.

We train on 1.6 million 32*32 color images that have been preprocessed by subtracting from each pixel its mean value over all images and then dividing by the standard deviation of all pixels over all images.

I've trouble distinguishing between "from each pixel its mean value over all images" and "standard deviation of all pixels over all images".

Since, I'm dealing with color images, can I just take rgb values of each pixel as one value or should I calculate the mean and SD for every color differently?

For example if I have r=255, g=255, b=255, can I take pixel value as (in binary), (r<<16)+(g<<8)+b ?

$\endgroup$

2 Answers 2

2
$\begingroup$

I believe that you will want to process each channel separately over all of the images. Otherwise a mean and variance will have very little meaning. And if you convert to grayscale before training, then the network can only retrieve grayscale images, which doesn't seem to match with what the paper describes.

I've trouble distinguishing between "from each pixel its mean value over all images" and "standard deviation of all pixels over all images".

This processing will requires two passes:

On The first pass, you will compute the mean pixel values of each channel, and the variance over the entire set of pixels in a channel. When you are finished, you should have 3075 values: one mean value per pixel per channel (32*32*3=3072), and one variance per channel (3).

On the second pass you will modify the images by taking the subtracting from each pixel the mean you found in the first pass, and dividing by the standard deviation from the first pass.

Let $\mu_r,i,j$ be the mean value of the red channel at the $i,j$-th pixel, and let $\sigma_r^2$ be the variance of the red channel over all pixels in all images. Then the new value corresponding to the red channel of the $i,j$-th pixel in the $k$-th image $r_{i,j,k}$ becomes

$$ \tilde{r}_{i,j,k} = \left(r_{i,j,k} - \mu_{r,i,j}\right)/\sqrt{\sigma_r^2}$$

Where

$$\mu_{r,i,j} = \frac{1}{K}\sum_k r_{i,j,k}$$ $$\sigma_r^2 = \frac{1}{IJK-1}\sum_{i,j,k} r_{i,j,k}^2 - \frac{1}{IJK-1}\sum_{i,j,k} r_{i,j,k}$$

This is my interpretation of the paper. However, since the focus of the paper is on autoencoders and deep belief networks, the image preprocessing step is only mentioned in passing. Like most academic papers, it gives enough details for the reader to understand what was done without necessarily giving enough details to reproduce the result without lots of reading between the lines.

Good luck to you!

$\endgroup$
3
  • $\begingroup$ Thanks.. For my sake, can you expand sigma-r part? Over all pixels over all images, means calculating mean over all pixels linearly over all images and use it in calculating SD? $\endgroup$ Oct 13, 2013 at 9:43
  • $\begingroup$ Sure. I have made an edit to my answer, let me know if it helps. $\endgroup$
    – nispio
    Oct 14, 2013 at 17:57
  • $\begingroup$ Why would there need to be a mean value per pixel per channel. Why not just a mean per channel over all images? $\endgroup$ Feb 8, 2019 at 6:40
3
$\begingroup$

For example if I have r=255, g=255, b=255, can I take pixel value as (in binary), (r<<16)+(g<<8)+b ?

While this number would identify the colour, it would not help with a mean or standard deviation. e.g. pure blue would be 255 but pure red would be 16711680

The paper is unclear so you have a few choices:

  1. Process each channel individually
  2. Convert the image to grayscale, find the mean and std dev, and use this for each channel.

There may be other methods that use different colour spaces.

$\endgroup$
1
  • $\begingroup$ +1 Thanks for your time.... but paper says 3072 visible units 3072=32*32*3, it seems 3 channels are taken. It seems 3 channels are taken individually. What do you think? $\endgroup$ Oct 12, 2013 at 15:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.