What is the effect of noise on Shannon entropy?

Question

Since entropy is a measure of uncertainty or randomness, intuitively we would suppose that adding noise to an image would increase its entropy since we are now more uncertain about the information of the image. I actually confirmed (partially) this assumption with the following MATLAB code where i noticed that when adding gaussian noise to a grayscale image, the entropy increases, whereas adding salt & pepper noise slightly decreases the entropy. Why is this happening and how does noise affect overall the Shannon entropy of an image?

This is my MATLAB code:

load('trees.mat');
RGB=ind2rgb(X,map);
GRAY=rgb2gray(RGB);
vecRGB=RGB(:);                                 %--vector of RGB image
vecGRAY=GRAY(:);                               %--vector of GRAYSCALE image
HVECR=entropy(vecRGB);                         %--entropy of RGB image
HVECG=entropy(vecGRAY);                        %--entropy of GRAYSCALE image
GRAY_GAUSSIAN=imnoise(GRAY,'gaussian');        %--add gaussian noise to GRAYSCALE image
GRAY_SALT=imnoise(GRAY,'salt & pepper');       %--add salt & pepper noise
vec_GRAYG=GRAY_GAUSSIAN(:);                    %--vectorize image with gaussian noise
vec_GRAYSP=GRAY_SALT(:);                       %--vectorize image with salt & pepper noise
HVECGG=entropy(vec_GRAYG);                     %--entropy with gaussian noise
HVECGSP=entropy(vec_GRAYSP);                   %--entropy with salt & pepper noise

And these are the entropies i got:

• Entropy of Grayscale Image (no noise)=5.4723
• Entropy of Grayscale Image (Gaussian Noise)=7.6948
• Entropy of Grayscale Image (Salt & Pepper Noise)=5.4380

Answer 1

The grayscale image is actually a discrete image. entropy calculates a histogram and from that extracts "empiric" probabilities, to be used in the common information formula

$$I= -\log_2 (P(g))$$

with $P(g)$ being the (empiric) probability of an intensity value $g$.

Now, entropy $H$ is the expectation of Information. For your whole image $V$ that means

$$\begin{align} H(V) &= E\left\{ I(g_{x,y})\right\}\\ &= \sum\limits_{x,y\in \text{idx(V)}} P(g_{x,y})\cdot\left(-\log_2 (P(g_{x,y}))\right)\\ &= -\sum\limits_{x,y\in \text{idx(V)}} P(g_{x,y})\log_2 (P(g_{x,y})) \end{align}$$

If you look at the product in the sum intensely enough, you'll notice that the sum will take a maximum value if all properties are equal – that is, if every intensity value is just as likely as every other. The histogram of that image would be a flat line.

Adding "well-behaved" noise will make almost any image look more like that – in the end, pixels with identical intensity will be relatively common in most natural images, and adding noise will "break" that.

Now, salt'n'pepper is different: you add extreme values to your image. That will make the intensity at many pixels clip at the maximum intensity (or the minimum one), skewing the histogram away from flat to a very "horned" thing, which has less entropy.