Shannon capacity limit & FEC comparisons

Question

In this figure from Fundamentals of Digital Communications by Madhow , pp 328 there is a comparison of a rate 1/3 turbo code with the shannon limit for rate 1/3 BPSK:

Trying to understand the limit computation - I start with BPSK achieving 1 bits/sec/Hz over passband AWGN. Factoring in 1/3 rate coding this becomes 0.333 bits/sec/Hz.

Finally, solving the capacity equation $E_b/N_0 = \frac{2^{C/B} - 1}{C/B}$ for $C/B = 0.333$ gives us $E_B/N_0 = \frac{2^0.333 - 1}{0.333} = 0.7797$ which in dB is -1.08 dB. So about 1/2 dB off.

Looking at the capacity curve below it appears -0.58 dB corresponds to 0.648 bits/sec/Hz - it's unclear to me what the significance of this number is...

Edit: Per @AlexTP's note I plotted the binary awgn capacity. First for R=1, and second for R=0.333. I'm not really sure how to interpret either for the bound shown in my original figure...

Answer 1

I start with BPSK achieving 1 bits/sec/Hz over passband AWGN. Factoring in 1/3 rate coding this becomes 0.333 bits/sec/Hz.

This is not correct.

The Shannon noisy channel coding theorem states that the reliable discrete-time rate $r$ (whose unit is bits per symbol, or bits per channel-use, or bpcu) is upper-bounded

$$r \lt \frac{1}{2}\log_2\left(1 + \frac{S}{N} \right) \tag{1}$$

where $S$ and $N$ are the discrete-time symbol energy and noise energy respectively.

Call $R_s$ the symbol rate (symbol per second), we define the rate $R$ (bits per second) as

$$R = R_s r \lt \frac{R_s}{2}\log_2\left(1 + \frac{S}{N} \right)\tag{2}$$

Because $r=\frac{S}{E_b}$, $r=\frac{R}{R_s}$, and $N=\frac{N_0}{2}$, $$\frac{S}{N}=\frac{E_b}{N_0}\frac{2R}{R_s}\tag{3}$$

From (2) and (3) $$\frac{R}{R_s/2} \lt \log_2\left(1 + \frac{E_b}{N_0} \frac{R}{R_s/2} \right) \tag{4}$$

As $R_s/2$ is the infimum of required bandwidth (Nyquist sampling theorem), we call $\eta = \frac{R}{R_s/2}$ the supremum of spectral efficiency, and

$$\frac{E_b}{N_0} \gt \frac{2^\eta - 1}{\eta} \tag{5}$$

Remind that $\frac{R}{R_s}=r$,
$$\eta = 2 r \tag{6}$$

For a rate 1/3 turbo code, there is 1/3 info bit every coded bit. As we use BPSK, 1 coded bit is transmitted on 1 symbol. Therefore, $r=1/3$, and $\eta=2r=2/3$ and the Shannon limit is $-0.5497\textrm{dB}$ (close enough to -0.58, right?).

However, as I said in the comment section, the comparison to this limit does not make sense as this is a binary input channel implied by the BPSK modulation. The discrete-time capacity for such a channel assuming a uniform input distribution is derived in https://gdurisi.github.io/fbl-notes/bi-awgn.html (being discrete-time model, the power input of the evaluation routine is $\frac{S}{N}$, not $\frac{E_b}{N_0}$. To put it differently, it is (1)). As the capacity is not closed-form, we resort to either numerical evaluation or bounds to calculate the infimum $\frac{E_b}{N_0}$.

Let's fix $\eta = 2/3$ and evaluate the bi-AWGN capacity at $\frac{E_b}{N_0} = -0.5497\textrm{dB}$, the bi-AWGN capacity is about $0.3301 < 1/3$ bpcu. This is expected as the Shannon limit is attained at capacity-achieving distributions, which are (usually and probably) not the aforementioned uniform binary input one.

Answer 2

You've computed the bound on the capacity of an AWGN channel. But with the additional constraint that the signal must use BPSK modulation the capacity will be lower (I don't know offhand what it is but there should be a reference in the text).

Edit:

There's a good description of the difference between continuous- and discrete-input AWGN channels here:

Capacity of AWGN channel

You can see that the capacity of the BPSK+AWGN channel is lower than the capacity of the continuous-input channel. This means you won't be able to reproduce the -0.58 dB bound value in the figure 7.12 using the continuous-input capacity equation.

Equation 3.6 in the link provided by AlexTP and the graph in the answer above should give you the right capacity (although there seems to be a discrepancy in the units somewhere).