3
$\begingroup$

I understand the concept of channel capacity as the maximal rate of the channel code I can apply without making a mistake in the receiver, in that sense the capacity is between 0 and 1. What I don't understand is the meaning of the capacity in the AWGN channel case where it is calculated by C=(1/2)*log2(1+SNR) where clearly I get a number greater than 1 when the SNR is greater than 3 (linear scale). How can I pass more than 1 bits/channel use when the SNR is very high?

Thanks

$\endgroup$
  • 1
    $\begingroup$ The capacity is not between 0 and 1, and the channel symbols don't have to be binary. $\endgroup$ – John Jan 25 '14 at 17:50
  • $\begingroup$ @John : Ahmm..so the capacity actually tells me how many signalling levels I can use (BPSK,QPSK,16QAM,etc.) if my input is gaussian (which is never) even though I haven't specified a bandwidth? So what is the meaning of this quantity when BW is specified (W*log2(1+SNR))? $\endgroup$ – Gabe Jan 25 '14 at 19:00
  • 1
    $\begingroup$ Capacity is the maximum information rate (in bits per second) at which error-free communication is possible. It does not tell you how many signaling levels to use. $\endgroup$ – John Jan 25 '14 at 19:04
  • $\begingroup$ @John : but if I find that the capacity is e.g. 2.3 [bits/channel use] doesn't it tell me that I can use QPSK modulation because the channel can work (with appropriate coding) with above 2 bits per sample without error? $\endgroup$ – Gabe Jan 25 '14 at 20:11
  • $\begingroup$ Capacity is measured in information bits per second. I'm not familiar with bits/channel use. Anyway, the bound is for all possible symbol schemes. $\endgroup$ – John Jan 25 '14 at 21:03
4
$\begingroup$

There are two different kinds of channel models that are being confused here.

In the binary symmetric channel, the inputs and the outputs are constrained to be $0$ or $1$ and the key parameter is the transition probability: the probability that an input $0$ is changed to an output $1$ or vice versa. The channel capacity $C$ is a number between $0$ and $1$ and the meaning is that by using a suitable code of rate $R < C$ on the channel, we can transmit at a rate of $R$ information bits per channel use with arbitrarily high reliability (that is, with arbitrarily small bit error probability. A (block) code for such a channel would be a set of $M$ binary $n$-tuples that will convey $\log_2 M$ bits in $n$ channel uses at a rate $R = \frac{\log_2 M}{n}$. Usually $M$ is chosen as a power of $2$, say $2^k$ for some $k < n$. Note also that the binary symmetric channel capacity cannot exceed 1 bit per channel use.

The discrete-time Gaussian channel is a different beast. The inputs can be arbitrary real numbers (it is not necessary that they be restricted to $0$ or $1$) and if the $i$-th input is $x_i$, then the $i$-th output is $x_i+n_i$ where the $n_i$ are independent zero-mean Gaussian random variables with variance $\sigma^2$. $x_i^2$ is called the power in the $i$-th input and $\sigma^2$ is called the noise power. A (block) code for such a channel is a collection of $M$ $n$-vectors $\mathbf x^{(i)}$ (called a codebook) that can be used to transmit $\log_2 M$ bits in $n$ channel uses. Various realistic constraints are placed on the code. One such constraint is that the peak power is bounded by $P$: the magnitudes of all the $M\times n$ entries $x_j^{(i)}$ in the codebook are no larger than $\sqrt{P}$. Another is that that the average power is bounded by $P$: $$\frac{1}{Mn}\sum_{i} ||\mathbf x^{(i)}||^2 = \frac{1}{Mn} \sum_i \sum_j \left|x_j^{(i)}\right|^2\leq P.$$ The capacity of the channel depends on the signal to noise ratio $P/\sigma^2$ and it can exceed one bit per channel use. Think of it this way: we can use multilevel pulse amplitude modulation to transmit more than one bit per channel symbol on this channel whereas with the binary symmetric channel we are restricted to binary modulation and so cannot exceed one bit per channel use.

For lots of details on this kind of stuff at a level lower than a typical graduate text, see the following links (lecture notes of mine).

Lecture 6, Lecture 7, Lecture 8, Lecture 9, Lecture 10, Lecture 11, Lecture 12

$\endgroup$
3
$\begingroup$

There are two main factors when figuring out how many bits are transmitted per symbol (or "channel use"): the modulation and the error correction encoding. For instance, BPSK modulation with no encoding transmits 1 bit/symbol, while QPSK with no encoding transmits 2 bits/symbol. Higher order modulation schemes (e.g. 8-PSK, 16-QAM, 32-QAM, etc.) can transmit even more bits/symbol.

Error correction encoding reduces the number of information bits/symbol, because some of the bits are used for the error correction code. While this is unfortunate it is necessary to achieve arbitrarily low error rates without requiring insanely high SNR's.

A fairly common set of error correction codes is $\frac{1}{2}$ rate convolutional codes, which are called "$\frac{1}{2}$ rate" because half of the bits are information (i.e. payload) bits, while the others are code bits (often called "parity bits"). Thus, if you used QPSK and a $\frac{1}{2}$ rate convolutional code you would have an overall information transmission rate of $2*\frac{1}{2} = 1$ bit/symbol.

So, getting to your question, you ask- "but if I find that the capacity is e.g. 2.3 [bits/channel use] doesn't it tell me that I can use QPSK modulation because the channel can work (with appropriate coding) with above 2 bits per sample without error?"

No, that's not what it's saying. What it is saying is that there exists a modulation and error correction scheme (and please note that it does not say anything about what the appropriate modulation type[s] and/or error correction scheme[s] might be) that can get you reliably error free data transmission at up to 2.3 bits/symbol. Now, having said that, since QPSK has a data rate that is less than 2.3 bits/symbol, if you paired it up with an appropriate error correction scheme for your channel (for instance, are the errors bursty or non-bursty?) then you could probably get reliable communication through that channel.

The best codes that we know of for getting close to the Shannon limit are Turbo codes and low-density parity check codes (also known as Gallager codes).

$\endgroup$
  • $\begingroup$ great answer! I still get confused about units. In this thread we see [bits/channel use]. In books we see [bits/sec] and [bits/dimension]. What does [bits/channel use] mean? Thx. $\endgroup$ – John Jan 26 '14 at 1:26
  • $\begingroup$ @John bits/channel means "bits per individual transmission". In normal signal processing terminology it is exactly equal to "bits/symbol"- i.e. a symbol is one "channel use". $\endgroup$ – Jim Clay Jan 26 '14 at 1:48
  • $\begingroup$ thanks. So if we take the familiar voice grade telephone channel with 35 dB SNR and 3000 Hz BW, we find the AWGN capacity C is about 34.8 kbps which is a familiar number. To get from there to bits/symbol we divide by the symbol rate, which must be something close to 3000 giving 11 bits/symbol. $\endgroup$ – John Jan 26 '14 at 2:04
  • $\begingroup$ @John That is essentially correct, though the 3 kHz of bandwidth is one-sided bandwidth which means that there is really 6 kHz of two-sided bandwidth, which means that you should be able to get around 6000 symbols/s. $\endgroup$ – Jim Clay Jan 26 '14 at 5:10
  • $\begingroup$ Are you sure about the 6000? The telephone channel is bandpass, and the V.34 symbol rate is about 3400 Hz. $\endgroup$ – John Jan 26 '14 at 11:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.