# value of 0 log0 in entropy formula

Why is the value of $$p_i\log(p_i)$$ in entropy formula considered $$0$$ when $$p_i =0$$? I get that it is the limiting value, but does that mean $$p$$ is never equals to zero, but only tends to?

• If a symbol has probability zero, then it does not influence the calculation and there is no need to include it, so you never actually calculate $$0\log(0)$$.

• If you insist in including symbols with zero probability, then $$0\log(0)=0$$ by convention.

• it doesn't have to be "by convention". the limit $$\lim_{p \to 0} p \log(p) = 0$$ Mar 26, 2021 at 16:33
• @robertbristow-johnson I would agree only if you write the limit with $p \rightarrow 0^+$. But the limit itself (the "two-sided limit") does not exist, as far as I understand these things.
– MBaz
Mar 26, 2021 at 17:12
• yeah you're right. too late to change it. oops. Mar 26, 2021 at 19:03

I am not adding a lot to MBaz, mostly graphical hints. It can be interesting to look at the elementary function behind Shannon entropy: $$H: p\mapsto -p \log p -(1-p) \log (1-p)$$, displayed below:

While it seems not defined at $$p=0$$ or $$p=1$$, the function $$H$$ is very symmetric and behaves quite well at $$0$$ and $$1$$ for a Bernoulli trial. Here, we talk about a binary event ($$X$$ happens with probability $$p=P(X)$$. The converse event is "not $$X$$". While entropy is often described as a measure of information, it can be seen as a measure of uncertainty. If $$X$$ is always equal to 1, it is certain. If $$X$$ never occurs, its converse is certain as well. In both cases, we have not surprise: the uncertainty is zero, and the "definitions" $$-p \log p = 0$$ for $$p=0$$ or $$-(1-p) \log (1-p) =0$$ for $$p=1$$ make sense.

More details can be found in Entropy is a measure of uncertainty.