What is relationship between entropy and frequency in image

Question

When I was studying spatial frequency and entropy, it occured to me that "is there a relationship between entropy and frequency?".

As you can see, if we consider only horizontal line, the frequency of (a) is lower than (b).

Intuitively, I think that magnitude of frequency is proportional to entropy. But Google said nothing. Am I right?

Each entropy is the following.

$$H_a = -0.842\log_20.842 -0.157\log_20.157 = 0.209 + 0.419 = 0.628$$ $$H_b = -0.6\log_20.6 -0.4\log_20.4 = 0.442 + 0.529 = 0.971$$

(a)

(b)

Answer 1

If I understand correctly, you are taking the entropy in the image from the distribution of black and white pixels. In image (a) you have 50% black, 50% white, so the entropy is 1 bit. In image (b) you have 85% black, 15% white, so the entropy is 0.6 bits.

If you make another image (c) with black and white stripes equal in width to the black stripes in image (a), you will have 50% black, 50% white again, i.e. entropy 1. But the frequency will be less than image (a).

So for this notion of entropy, the answer is: No, in general there is no relation between entropy and frequency in an arbitrary image.

However, in real world images energy is more concentrated in low frequencies than in high frequencies. So if you define entropy in terms of the distribution of such images, you will find a relationship between frequency and entropy.

Answer 2

Spatial Frequency in an image can be looked at from two perspectives :-

• Global :- This comes through the notion of Fourier Transform which gives a global description of the frequency content of an image.
• Local :- This comes through the idea of Time-Frequency Analysis which aims at giving a local description of the frequency content of an image.

If you talk about the global frequency content and global entropy, then there exists no relationship between them (this is the best we know as of yet)

If you talk about the local frequency content, then a relationship can be observed. Infact this relationship turns out to be exactly the one you are alluding to:-
Frequency Content $\varpropto$ Entropy (Local Of Course!!!)

A very simple explanation to this is the following :-

Entropy is proportional to disturbance. Greater the variations in a local path of an image, greater will be the frequency content of that patch (follows easily from the definition of frequency) . In exactly the same way the entropy of the patch would also be higher.

In a smooth region of an image where there is absolutely no imaging variations, entropy will be $0$ and so will be the frequency.

Hopefully this answers your question.