I have a question about the Shannon Theorem. According to this theorem the maximum capacity of a link can be calculated with the formula: $$C=B\log_2(1+S/N)$$ where $B$ is the bandwidth and $S/N$ is the signal to noise ratio. But I am not very sure which is the meaning of the "capacity" term in this equation, since it should include the probability of bit error in same way.

What I mean is that the capacity might not be defined as "number of bits transmitted per second" but as something like "number of bits transmitted and correctly received per second". But it still makes no sense, since you can transmit with low $S/N$ and symbol length tending to $0$, getting a capacity that tends to infinite and a probability of bit error close to 0.5 (it is theoretically imposible to have more than 50% of error rate), so the "number of bits transmitted and correctly received per second" would tend to infinite and it would be higher than the Shannon limit. I was thinking that the right definition of the $C$ in the formula could be "number of bits transmitted per second $\times$ (0.5- Probability of bit error)", which could have sense but I am not very sure which is the real meaning of this measure.

  • $\begingroup$ a better, more general relationship, when $S$ and $N$ are functions of frequency (i think we call it "power spectral density" or "PSD") is $$ C= \int\limits_{0}^{B} \log_2\left(1 + \frac{S(f)}{N(f)} \right) \, df$$ $\endgroup$ – robert bristow-johnson Jan 22 '18 at 6:36

The formula you wrote corresponds to the error-free capacity. Namely, Shannon is telling us that $C$ is the maximum (theoretical) limit one can achieve.

Shannon managed to prove that if the information rate on the channel being used is less than $C$, then it is theoretically possible to find a coding system for the signal such that the transmission will have an arbitrarilly small probability of error. Unfortunately, the theorem does not tell us how to find that code, just that it exists.

That's why there is no reference to error rate in the formula for capacity. The formula itself is telling you the maximum information rate you can have with an arbitrary small amount of errors, and that value corresponds to $C$.

Sidenote: remember that the formula is only valid for white Gaussian noise.

| improve this answer | |
  • $\begingroup$ Just a nitpick: the theorem says that if the rate is less than $C$, then you can find a code such that the probability of error $\varepsilon>0$ is as small as you want -- but not zero. $\endgroup$ – MBaz Jan 21 '18 at 16:45
  • $\begingroup$ @MBaz True, I've edited the answer. $\endgroup$ – Tendero Jan 21 '18 at 16:59
  • $\begingroup$ white Gaussian noise has infinite power. i don't think you mean $N=\infty$. $\endgroup$ – robert bristow-johnson Jan 22 '18 at 6:31
  • $\begingroup$ and i am not sure your sidenote is valid anyhoo. $\endgroup$ – robert bristow-johnson Jan 22 '18 at 6:38
  • $\begingroup$ @robertbristow-johnson $N$ would be the variance of the noise (or the average power). Therefore, it is not infinite. The fact that the formula is valid for AWGN is correct. You can check that in Wikipedia. $\endgroup$ – Tendero Jan 22 '18 at 12:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.