Is there a relationship between the Shannon entropy of image and the output of the 2D Fourier transform (DFT) of the image?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Could you please review my question? If it answers your question, please mark it. $\endgroup$– RoyiCommented Apr 9, 2023 at 9:34
-
$\begingroup$ Perhaps a related question would be «does structure and correlation in the other domain»? $\endgroup$– Knut IngeCommented Jul 5 at 6:08
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
3
Entropy, in the information theory sense, is a measure of the amount of information that a signal contains.
The FFT, in a purely mathematical sense, neither adds nor destroys information -- it just transforms the signal from one domain into another.
So, barring practical difficulties such as numerical effects during computation, the amount of information -- and therefor the entropy -- of the image must remain unchanged.
answered Oct 15, 2022 at 0:41
-
1$\begingroup$ What I don't know is whether this equality of entropy is evident computationally -- I'm pretty sure that there is no way of calculating the actual entropy of an arbitrary signal. $\endgroup$ Commented Oct 15, 2022 at 0:42
-
$\begingroup$ I am not in agreement with your answer. Entropy, as in information theory, is a measure of the frequency of the values, not their energy. In Physical systems Entropy has to do with energy but because it changes the frequency of some values, not just because of the energy. $\endgroup$– RoyiCommented Jan 11, 2023 at 7:14
-
$\begingroup$ I'm not sure what Last-October-Me was thinking, but I'm pretty sure he would have agreed that "information", not "energy" was the correct word there. $\endgroup$ Commented Jan 11, 2023 at 15:58
$\begingroup$
$\endgroup$
Entropy calculation has to do with the frequency of the values of the data (Or the probability of values in case we have access to the generator itself).
For instance, take an image, then scramble its pixels locations (Not values), it has the same entropy though it clearly lost its visual meaning.
On the other side, the DFT (FFT is just a calculation method) has a lot to do with the spatial order of data. So in principle I can change a lot the DFT of the image by permuting the pixels yet no effect on its entropy, which means they have little to do with each other.
Yet, we can talk about some properties in the edge cases.
For instance, for a bounded values the maximum entropy is when the data has uniform distribution. This is exactly what Histogram Equalization is doing. Maximizing entropy which results, in many cases, in better contrast.
Usually when we stretch the contrast we have sharper edges which means higher frequency content. So we can say that in most cases higher entropy images will result with more energy in high frequency.
In the extreme case, where the image is a random uniform noise on the range [0, 255] for UINT8 data, it will have no spatial correlation and hence will have a DFT which is approximately uniform all over the spectrum.
$\begingroup$
$\endgroup$
There is obviously a relationship between the amount of information stored in an image (entropy) and the amount of structure in a corresponding Fourier spectrum. Let's consider a corner case: white noise that is "flat". This is an equivalent of a lower bound for entropy. The higher bound is unclear to me. As far as I know, this is an open research topic.
