12
$\begingroup$

I hope this question is appropriate for this site.

I came across this passage in The Three Body Problem, a novel by Liu Cixin:

The professor had put up two pictures: One was the famous Song Dynasty painting Along the River During the Qingming Festival, full of fine, rich details; the other was a photograph of the sky on a sunny day, the deep blue expanse broken only by a wisp of a cloud ... The photograph's information content - its entropy- exceeded the painting's by one or two orders of magnitude

Representative pictures:

Here is the painting Blue sky Is this true? How does one explain this counterintuitive phenomenon?

$\endgroup$
  • $\begingroup$ Is there more context in the book? $\endgroup$ – endolith Jun 9 '17 at 19:13
  • $\begingroup$ @endolith no, unfortunately not. $\endgroup$ – RSS Jun 9 '17 at 19:22
  • $\begingroup$ I wish entropy was the only measure of the content. But no. RGB Images are created for humans to look at, both paintings and photographs. So look at it yourself. Which one do you think is more informative and rich? Your choice is correct, regardless the computer measures we invent. $\endgroup$ – Tolga Birdal Jun 9 '17 at 19:32
  • $\begingroup$ @TolgaBirdal Fair enough, but I would still be interested to understand why the computers are getting it wrong in this case. $\endgroup$ – RSS Jun 9 '17 at 19:41
12
$\begingroup$

It depends how you define the term "information" or "entropy".

The conventional definition of entropy of an image is to think an image as a two dimensional matrix of pixel and $$H = - \sum_k p_k \log_2(p_k)$$ where $p_k$ is the probability, which is calculated from histogram, associated with gray level $k$.

This kind of entropy is correct if we ignore the correlation between pixels. For example the two images have the same entropy by this definition.

enter image description here enter image description here

It is not true if the correlation between pixels is considered. For example if the color first pixel in top-left has probability $p_k$, the next pixel surely has the same color and its color does not have the same probability $p_k$.

We human being, with you as an example, use this kind of correlation to perceive the images. This kind of correlation are called "details", and we/you expect that rich-detail images should have more information/entropy than simple ones. This is the reason why you have found it counterintuitive.

PS:

I tried calculating the entropy of the two images you have posted, but they are not different "by one or two orders of magnitude" !!!!

"Along the River During the Qingming Festival" entropy about 7

"The sky" entropy about 6

They must not be the same files of the professor.

$\endgroup$
  • $\begingroup$ Thanks, I think this is the answer I was looking for. Of course the images I uploaded were meant to be representative only, I have no idea of what the fictional professor actually showed to the class :D $\endgroup$ – RSS Jun 9 '17 at 20:06
1
$\begingroup$

First of all, its not the painting itself but the photograph (or a scan) of it that we can compare against the photograph (or the scan) of something else, such as a natural scene.

Based on your provided images, perceptually speaking the painting should of course involve more information compared to a simple sky. The result is that when compressed, the painting-file will be larger than sky-file under same compression algorithm.

That being said, however, the simple sky scene might include perceptually invisible components such as quantization artifacts, the color gradient, or similar things, which, eventhough you cannot perceive their existance, a mathematical algorithm will still treat as statistical information so that the entropy bound of the image is increased. Resulting in a larger file.

The same could of course be happening for the painting file as well.

$\endgroup$
  • $\begingroup$ You've raised a good distinction i.e. did the professor compare a photograph with the actual painting (let's call this the weaker hypothesis) or would even a scan of the painting contain less info (stronger hypothesis). So, as per your explanation, only the weaker hypothesis is true? $\endgroup$ – RSS Jun 9 '17 at 19:39
  • $\begingroup$ I used the terms photo and scan to denote an $N$-bit digitized sequence $f[n1,n2]$ which represents the image information plus some distortion due to digitzation process. The truth is perceptually speaking painting contains more information. But the mathematical concept of entropy is fundamentally a statistical measure of information based on probabilities. So invisible pixel variations would still be considered as information, and will be coded, unless discarded by the quantizer of a perceptual codec such as jpeg variants.. $\endgroup$ – Fat32 Jun 9 '17 at 19:50
0
$\begingroup$

Both contain same information i.e both have 1 bit of information . Consider on board level there is 2 two images one of painting and other photograph . So probability of one image is 1/2 = 0.5 . As you don't know which one is the image before seeing them.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.