# What the entropy equations mean?

In wavelet packet image compression, different types of entropy methods can be used, like Shannon and log-energy.

• Shannon entropy uses this equation $\mathrm{ent}= -\sum (x^2 \times \log(x^2))$, where log-energy equation is $\mathrm{ent}=\sum(\log x^2)$, what this equation means?
• which technique can achieve better results for images (in terms of PSNR)? and how can I determine the appropriate technique?
• I don't understand what you mean with "different types of entropy methods can be used": Entropy is a property of a stochastic signal source. you seem to be referring to a very specific kind of wavelet decomposition-based algorithm, but don't mention which one that is. A decomposition is a thing that you apply to a signal, a method. Entropy is not a method. – Marcus Müller Mar 9 '18 at 22:16
• @MarcusMüller In image processing, people use "entropy" in a different way than in communications. It's somewhat related to the information content of the image, but I don't understand it very well myself. – MBaz Mar 9 '18 at 23:28
• huh, OK; still, I feel a bit left out on what wavelet packet decomposition OP is talking about and would very much like to know where I can read about it. Gimme a buzzword. – Marcus Müller Mar 9 '18 at 23:56
• Entropy is used as a criterion to decide whether a wavelet packet branch should be further decomposed, or to prune a wavelet packet tree. This carries similarities with segmentation techniques like split-and-merge – Laurent Duval Mar 10 '18 at 0:10
• I recommend you to plot the expression inside the sum (-x^2 log(x.^2)). I assume that x are probabilities (or weights). so you would generate x in the range between zero and one. In the Shannon case you will see that probabilities around zero and one are assigned to very low values. So if you are looking for minimum entropy, these are cases which are favoured. Probabilities around 1/2, however, are assigned to the maximum value and so they are of less interest. – Irreducible Mar 12 '18 at 6:44

Entropy can be used as a criterion to decide whether a wavelet packet branch should be further decomposed, or to prune a wavelet packet tree. This carries similarities with segmentation techniques like split-and-merge.

Let us take one sign convention for entropy. One hopes that entropy diminishing, locally, across some levels, meas that this part of the image is simplified by one wavelet decomposition, and when the entropy starts to increase (or vice-versa), further decomposition levels become useless to simplification. Following the entropy gradient is a way to find some "optimal" wavelet packet splitting.

But first, optimal for what purpose is unclear from your question so far. Denoising, compression? Entropy can provide a nice guess to predict compression performance with an actual coder, but should be applied to binned/quantized wavelet coefficients, and may be limited unless you use higher-order/cross-entropy to take into account advanced vector coders (as compared to scalar ones).

Second, traditional wavelets packets for image are implemented separably, and notoriously, separable schemes and packets lack in efficient representation for piecewise regular images, and induce increased aliasing, that may yield mildly efficient decompositions.

Codes implementing such decompositions are presented in Matlab, see for instance: wpdec2: Wavelet packet decomposition 2-D, and the reference paper Coifman, R.R.; M.V. Wickerhauser (1992), “Entropy-based algorithms for best basis selection,” IEEE Trans. on Inf. Theory, vol. 38, 2, pp. 713–718.

• I use wavelet packet in image compression, i mean better in terms of image quality (PSNR). Which of Shannon and log energy entropy can achieve higher PSNR? – user24907 Mar 10 '18 at 16:51
• Honestly, I have no general response. This is a topic I am currently re-investigating. It depends a lot of the nature of the classe of images you consider – Laurent Duval Mar 10 '18 at 17:00
• I use Lena image, i'll be gratefull if you helped me with a source in this topic. – user24907 Mar 11 '18 at 9:15