Tag Info

• 11.8k

What is the entropy for these cases?

Adding a bit of detail to Marcus' answer: Your question is about the "entropy of symbols" and the "entropy of real numbers". In information theory, only sources have entropy. A source has an alphabet,...
• 13.8k
Accepted

What is the entropy for these cases?

The entropy $H(X)$ of a continuous random variable $X$ is infinite. Proof is trivial (note that we can, without loss of generality, use the natural logarithm, since any other logarithm is the same but ...
• 26.6k
Accepted

Mutual information of $\infty$-PAM

I got an answer here, https://math.stackexchange.com/questions/1554659/mutual-information-for-a-continuous-uniform-distribution The $\text{erf}(x)$ function approximates to 1 with an error less ...
• 171

Is concept of "bit" in computer programming similar to the concept of "bit" in information theory?

Bit is a unit of measurement and multiple quantities are measured in bits. It's not that bit in programming and information theory mean different things. It's that memory and information content ...
• 121

Only top scored, non community-wiki answers of a minimum length are eligible