Shannon capacity limit & FEC comparisons

In this figure from Fundamentals of Digital Communications by Madhow , pp 328 there is a comparison of a rate 1/3 turbo code with the shannon limit for rate 1/3 BPSK: Trying to understand the limit computation - I start with BPSK achieving 1 bits/sec/Hz over passband AWGN. Factoring in 1/3 rate coding this becomes 0.333 bits/sec/Hz.

Finally, solving the capacity equation $$E_b/N_0 = \frac{2^{C/B} - 1}{C/B}$$ for $$C/B = 0.333$$ gives us $$E_B/N_0 = \frac{2^0.333 - 1}{0.333} = 0.7797$$ which in dB is -1.08 dB. So about 1/2 dB off.

Looking at the capacity curve below it appears -0.58 dB corresponds to 0.648 bits/sec/Hz - it's unclear to me what the significance of this number is... Edit: Per @AlexTP's note I plotted the binary awgn capacity. First for R=1, and second for R=0.333. I'm not really sure how to interpret either for the bound shown in my original figure...  • The author must have used the capacity for *binary-input AWGN" channels. The details can be found here gdurisi.github.io/fbl-notes/bi-awgn.html (Equation 3.6). However, I don't know how the author came up with the limit in the cited figure. Typo maybe?! Jul 16 '21 at 11:55
• @AlexTP I made an edit with the plot of eq 3.6 - does this look correct to you? I also don't immediately understand why the typical shannon capacity curve would have an asymptote at -1.6 dB Eb/N0 but this curve does not? Jul 16 '21 at 18:13

I start with BPSK achieving 1 bits/sec/Hz over passband AWGN. Factoring in 1/3 rate coding this becomes 0.333 bits/sec/Hz.

This is not correct.

The Shannon noisy channel coding theorem states that the reliable discrete-time rate $$r$$ (whose unit is bits per symbol, or bits per channel-use, or bpcu) is upper-bounded

$$r \lt \frac{1}{2}\log_2\left(1 + \frac{S}{N} \right) \tag{1}$$

where $$S$$ and $$N$$ are the discrete-time symbol energy and noise energy respectively.

Call $$R_s$$ the symbol rate (symbol per second), we define the rate $$R$$ (bits per second) as

$$R = R_s r \lt \frac{R_s}{2}\log_2\left(1 + \frac{S}{N} \right)\tag{2}$$

Because $$r=\frac{S}{E_b}$$, $$r=\frac{R}{R_s}$$, and $$N=\frac{N_0}{2}$$, $$\frac{S}{N}=\frac{E_b}{N_0}\frac{2R}{R_s}\tag{3}$$

From (2) and (3) $$\frac{R}{R_s/2} \lt \log_2\left(1 + \frac{E_b}{N_0} \frac{R}{R_s/2} \right) \tag{4}$$

As $$R_s/2$$ is the infimum of required bandwidth (Nyquist sampling theorem), we call $$\eta = \frac{R}{R_s/2}$$ the supremum of spectral efficiency, and

$$\frac{E_b}{N_0} \gt \frac{2^\eta - 1}{\eta} \tag{5}$$

Remind that $$\frac{R}{R_s}=r$$,
$$\eta = 2 r \tag{6}$$

For a rate 1/3 turbo code, there is 1/3 info bit every coded bit. As we use BPSK, 1 coded bit is transmitted on 1 symbol. Therefore, $$r=1/3$$, and $$\eta=2r=2/3$$ and the Shannon limit is $$-0.5497\textrm{dB}$$ (close enough to -0.58, right?).

However, as I said in the comment section, the comparison to this limit does not make sense as this is a binary input channel implied by the BPSK modulation. The discrete-time capacity for such a channel assuming a uniform input distribution is derived in https://gdurisi.github.io/fbl-notes/bi-awgn.html (being discrete-time model, the power input of the evaluation routine is $$\frac{S}{N}$$, not $$\frac{E_b}{N_0}$$. To put it differently, it is (1)). As the capacity is not closed-form, we resort to either numerical evaluation or bounds to calculate the infimum $$\frac{E_b}{N_0}$$.

Let's fix $$\eta = 2/3$$ and evaluate the bi-AWGN capacity at $$\frac{E_b}{N_0} = -0.5497\textrm{dB}$$, the bi-AWGN capacity is about $$0.3301 < 1/3$$ bpcu. This is expected as the Shannon limit is attained at capacity-achieving distributions, which are (usually and probably) not the aforementioned uniform binary input one.

• I follow your equations to derive $\eta$ in terms of code rate but just to confirm if I followed this from my equation of $Eb/N0 = (2^{\eta}-1)/(\eta)$ where $\eta = C/B$ and normalizing for a symbol rate of 1, C=0.333 bits/sec, B=1/2 (min Nyquist bw) so I'd have $Eb/N0 = (2^{2/3} - 1)/(2/3)$ I think this matches what you have, but just want to confirm the thought process looks ok. Jul 18 '21 at 13:37
• @user67081 yes, but keep in mind that the rate $C=\frac{1}{3} \textrm{bps}$ comes from $B=\frac{1}{2}\textrm{Hz}$ implies symbol rate $R_s = 2B= 1 \textrm{baud}$; and by the Turbocode setting and BPSK modulation, $r=\frac{1}{3} \textrm{bpcu}$. Jul 18 '21 at 17:18
• thanks. One more question - could you explain why the binary awgn capacity curve does not have an asymptote like the shannon ultimate limit at -1.6 dB and instead seems to allow capacity > 0 for arbitrarily low Eb/N0? I'm confused why it does not exhibit the same behavior there Jul 21 '21 at 14:41
• @user67081 Yes, there must be. Intuitively, the infinimum EbNo of bi-AWGN is lower-bounded by the infinimum EbNo of AWGN and, therefore, lower-bounded by the Shannon limit -1.6dB. Jul 21 '21 at 15:17
• @user67081 The evaluation routine given by Durisi is for discrete time, not continous time. Specifically, the input of the routine is the SNR $S/N$, not EbNo. The two are not equivalent (see my Equation (3)) as you need to specify the symbol rate $r$, which has impact in the spectral efficiency. Differently put, no beautiful Equation (5) for bi-AWGN because no closed form capacity formula. Jul 21 '21 at 15:25

You've computed the bound on the capacity of an AWGN channel. But with the additional constraint that the signal must use BPSK modulation the capacity will be lower (I don't know offhand what it is but there should be a reference in the text).

Edit:

There's a good description of the difference between continuous- and discrete-input AWGN channels here:

Capacity of AWGN channel

You can see that the capacity of the BPSK+AWGN channel is lower than the capacity of the continuous-input channel. This means you won't be able to reproduce the -0.58 dB bound value in the figure 7.12 using the continuous-input capacity equation.

Equation 3.6 in the link provided by AlexTP and the graph in the answer above should give you the right capacity (although there seems to be a discrepancy in the units somewhere).

• I made some edits if you could take a look and see if this is what you're thinking of? Jul 16 '21 at 21:46