'bits' are used with 2 different meanings. One can use 'bits' to mean the binary digits i.e. 1's and 0's. Bits are also the units of information in a event of discrete source. I think when channel capacity is measured in bits/sec, we use the second meaning of bits. Not how many 1's and 0's we can send down the channel but how much information we can send per second through that channel. Is my interpretation right?

  • $\begingroup$ You are correct: it refers to information, not to 1s and 0s. $\endgroup$
    – MBaz
    Mar 22, 2016 at 2:14

1 Answer 1


in Information Theory, the amount of information in a message is proportional to the logarithm of the probability of occurrence of the message:

$$ I(m) = -C \log\left( P(m) \right) $$

the only difference between different bases of logarithms is a scaling constant. if $ C = \frac{1}{\log(2)} $, that is

$$ \begin{align} I(m) & = -\frac{1}{\log(2)} \log\left( P(m) \right) \\ &= - \log_2\left( P(m) \right) \\ \end{align} $$

then we say that the $I(m)$ is in units of bits. the reason why is if the message $m$ had only two possibilities and both had equal likelihood, then $P(m) = \frac12 $ and $I(m) = 1$. there are two messages, two possibilities of equal likelihood, like heads and tails of a coin. and you need only one bit, 0 or 1, to represent the two possibilities.

  • $\begingroup$ Thanks for the confirmation. That was bothering and confusing me for some time. Actually, if you need to send 1's (sharp rectangular pulses) and want to receive them as they are, you need infinite bandwidth. Finite bandwidth channels will have non-zero rise and fall times. $\endgroup$ Mar 22, 2016 at 2:19
  • $\begingroup$ we hadn't gotten to channel capacity and bandwidth yet. that's deeper into Information Theory. $\endgroup$ Mar 22, 2016 at 2:31
  • $\begingroup$ I have gone to channel capacity through mutual information though it is a bit to difficult to comprehend. In almost every book on digital communications, in the beginning itself Shannon's noise theorem is given. $\endgroup$ Mar 22, 2016 at 5:52
  • 1
    $\begingroup$ Consider this: a stream of $M$-bit words all with probability $2^{-M}$. That stream of words go into an ideal $M$-bit D/A converter and the output of that goes into an ideal $M$-bit A/D converter but there is some noise added. The sample rate is $2B$ where $B$ is the channel bandwidth. The signal coming from that stream of data has a power of $S$ and you can figure that out from the D/A spec of output from $-2^{M-1}$ to $2^{M-1}-1$. the added noise has power $N$, but make this assumption that the p.d.f. of the noise is also uniform p.d.f. of width 1 LSB (loudest it can be with no error). $\endgroup$ Mar 22, 2016 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.