1
$\begingroup$

The sampling rate says that one Hz can carry at least 2 samples, and the bit rate is obtained by multiplying the sampling rate by the number of bits per sample. Thus, many samples and many bits can be contained in one Hz. Shannon, on the other hand, says that the maximum bit rate that can be transmitted over a channel is twice the bandwidth. I am confused. Could you please help me?

$\endgroup$
10
  • 1
    $\begingroup$ "The sampling rate says that one Hz can carry at least 2 samples": no, it doesn't say that – this might be a misunderstanding! Where do you take that from? $\endgroup$ Nov 25 '20 at 9:41
  • 2
    $\begingroup$ yes, but "1 Hz carries at least 2 samples" is not a statement that makes a lot of sense to me – are you inferring some discrete information content to a continuous signal bandwidth? $\endgroup$ Nov 25 '20 at 10:00
  • 3
    $\begingroup$ Let's focus on the other claim, though: "Shannon says the maximum bit rate […] is twice the bandwidth": no, he doesn't: Shannon capacity very clearly allows for arbitrary much information per bandwidth given sufficiently high SNR $\endgroup$ Nov 25 '20 at 12:06
  • 1
    $\begingroup$ @Noha: please cite your sources so we can look at it together. It appears that you are misquoting or misunderstanding what you have read or "heard". $\endgroup$
    – Hilmar
    Nov 25 '20 at 12:25
  • 1
    $\begingroup$ @Noha Lathi is correct. Do you see the difference between what Lathi says and what you asked? $\endgroup$
    – MBaz
    Nov 25 '20 at 16:50
2
$\begingroup$

First let's estabish the context.

Let's say you want to transmit a sequence of numbers $\lbrace a_k \rbrace_{k=1}^{N} = a_1, a_2, \ldots$ Using PAM, this sequence is transmitted with the signal $$s(t) = \sum_{k=1}^N a_k p(t-kT_p),$$ where the pulses $p(t-kT_p)$ form an orthonormal set, that is, $$\int_{-\infty}^\infty p(t) p(t) \text{d}t = 1$$ (which implies that the pulse $p(t)$ has unit energy) and $$\int_{-\infty}^\infty p(t-jT_p) p(t-kT_p) \text{d}t = 0$$ whenever $k \neq j$, which allows the receiver to recover the transmitted sequence using a correlator or a matched filter.

Second, let's see what Nyquist and Shannon say.

First of all, the pulses in $s(t)$ are transmitted at a rate $R_p = 1/T_p$. This is called the "pulse rate" or the "symbol rate". You want $T_p$ to be short, to transmit information faster. However, this increases the bandwidth $B$ of $s(t)$. The relationship is $$B = \alpha \frac{R_p}{2} = \alpha \frac{1}{2T_p}.$$

The value of $\alpha$ is the "spectral efficiency" of the pulse $p(t)$. The minimum possible value is $\alpha=1$, when $p(t)$ is a sinc pulse. In this case, $B=R_p/2$ or $R_p=2B$. This is what it means to say that "$2B$ pieces of information can be transmitted per Hz of bandwidth available."

If this was all there was to it, then an infinite amount of information could be transmitted in any amount of time. All one has to do is let the $a_k$ be arbitrary real numbers, with infinite precision. Even if one is constrained to choose from a constellation, such as $M$-QAM, one could choose $M$ to be as large as needed.

In practice, however, noise, distortion, and other imperfections prevent transmission of $a_k$ numbers with infinite precision. The simplest model of a communications system assumes that the received signal $r(t)$ is corrupted with white Gaussian noise with power spectral density $N_0/2$: $$r(t) = s(t) + n(t).$$

Transmission under the effects of noise restrict the choice of symbols $a_k$: they must belong to a finite set whose elements cannot be easily mistaken for one another. For example, if we constrain the symbols to take values either $-0.5$ or $0.5$, and the noise magnitude is very unlikely to be larger than $0.5$, then transmission can be done very reliably; however, we pay a price in two different ways:

  • Power: since the symbol amplitudes cannot be made too small, a minimum amount of power is required;
  • Rate: since we're limited in the choice of symbol amplitudes, we can't transmit as fast as we wish.

In terms of a $M$-QAM constellation, the consequence is that we can't increase $M$ arbitrarily; when $M$ is made too large, the system cannot transmit information reliably any more.

This is where Shannon comes in. He was interested in the relationship between the entropy of the information source that produces the transmitted sequences $\lbrace a_k \rbrace$, the bandwidth, and the signal to noise ratio in the received signal. He definitely did not say that the maximum bit rate is twice the bandwidth; rather, the maximum bit rate is given by the bandwidth, the SNR, and some specific properties of the sequence $\lbrace a_k \rbrace$. The study of these properties is known as "coding theory".

Finally: is it a coincidence that a signal with bandwidth $B$ must be sampled at rate $2B$, and that a signal with bandwidth $B$ can transmit symbols at rate $2B$? It is indeed not a coincidence. You want to transmit symbols at rate $R_p$; in other words, you want a signal that has specific amplitudes at times $kT_p$:

enter image description here

Think of these amplitudes as signal samples:

enter image description here

The blue signal is the transmitted signal $s(t)$, and since its samples have rate $R_p$, then it has bandwidth $B=R_p/2$.

$\endgroup$
15
  • 1
    $\begingroup$ Right on both counts. Just note that, in order to approach Shannon's capacity, coding must be used. $\endgroup$
    – MBaz
    Nov 26 '20 at 0:05
  • 1
    $\begingroup$ @Noha Please see my edits. In brief: ** You are correct that if there is no noise, then the bit rate is infinite, and there is no limit on the number of bits per symbol. ** The relationship between sampling theory and pulse rate is explained at the end of my edited answer. $\endgroup$
    – MBaz
    Nov 27 '20 at 17:28
  • 1
    $\begingroup$ I think you are being confused by the vague sentence "pieces of information". What is transmitted at rate $2B$ are the symbols $\lbrace a_k \rbrace$. Assume the channel is noiseless and you want to transmit one hundred bits, say 110110... You can write this down as the fractional number 0.110110..., which you then convert to decimal. Let's say this number is $a_0$. Then, transmit the pulse $a_0 p(t)$, which requires bandwidth $B$. The receiver proceeds backwards, from $a_0$ to the information sequence 110110.... (continued) $\endgroup$
    – MBaz
    Nov 30 '20 at 1:37
  • 1
    $\begingroup$ This process works for any number of bits; in the noiseless channel, you can transmit an arbitrary amount of information with a single pulse. When there is noise, it becomes very hard to distinguish between different values of $a_k$, and this limits the amount of information that each pulse can carry. The maximum amount of information given $B$ and the SNR is given by Shannon. However, the number of symbols per second is always $2B$. $\endgroup$
    – MBaz
    Nov 30 '20 at 1:40
  • 1
    $\begingroup$ You are correct about the sampling and capacity theorems, and that they are unrelated. In that paragraph, Lathi is trying to show (in a non-rigorous way) that the baud rate is actually bounded by $2B$, and it does that by using an argument based on the sampling theorem, relating the required $2B$ samples per second and the $2B$ symbol amplitudes. This is probably not the best approach, and Nyquist himself didn't use this argument as far as I know. See a derivation of the $2B$ bound that does not use the sampling theorem here. $\endgroup$
    – MBaz
    Nov 30 '20 at 23:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.