Rectangular 8 QAM bit error rate calculation

Question

I am simulating an 8-QAM communication system over an AWGN channel for various signal to noise ratios, in order to plot the average bit error. However I have not been able to derive an equation for the QAM system in terms of the Q-function.

The 8-QAM system is rectangular with 2 possible values for the x basis vector and 4 possible values for the y basis vector such that the constellation consists of 8 symbols:

(-d/2,3d/2), (d/2,3d/2)
(-d/2,d/2), (d/2,d/2)
(-d/2,-d/2), (d/2,-d/2)
(-d/2,-3d/2), (d/2,-3d/2)

where d is the distance between symbols.

The symbols are coded using grey coding for the x and y component as follows:

(0,10), (1,10)
(0,11), (1,11)
(0,01), (1,01)
(0,00), (1,00)

My attempt to derive an expression for bit error rate was based on the symmetry of the constellation as it consists of only two unique error scenarios, since the received vectors are:

r1 = s1 + n1, r2 = s2 + n2

The probability of error in each scenario is $$ P(bit error scenario 1) = P(r1<0|s1) + P(r2>d|s1) + P(r2<0|s1) = 3Q(d/2\sigma) $$ $$ P(bit error scenario 2) = P(r1<0|s1) + P(r2<d|s1) = 2Q(d/2\sigma) $$ Average probability of bit error is then:

$P(bit error) = (5/2)Q(d/2\sigma)$

Assuming the transmit signals are equi-probable. and:

$d = \sqrt(2E_s/3), $ $\sigma =\sqrt(N_0/2)$

however there is significant disparity between this theoretical error rate and the simulated error rate. I would appreciate it if someone could point me in the right direction, thanks.

Answer 1

Neglecting square terms of the Q function, your expression is correct, but it is the symbol error probability, not the bit error probability:

$$P_s=2.5\,Q\left(\frac{d}{2\sigma}\right)\tag{1}$$

With Gray encoding, the bit error probability is closely approximated by

$$P_b\approx\frac13Ps\tag{2}$$

because there are $3$ bits per symbol, and the most likely symbol errors (i.e. between neighboring symbols) cause only one bit error.

Furthermore, if $N_0$ is the one-sided noise density, then $\sigma=\sqrt{N_0/2}$, and the energy per bit is $E_b=E_s/3$, where $E_s$ is the symbol energy. It remains to express the distance $d$ between the symbols in terms of $E_s$. The energy of the four inner symbols is $d^2/2$. The energy of the corner symbols is $5d^2/2$. So for equally probable symbols, the average symbol energy is

$$E_s=\frac12\cdot\frac{d^2}{2}+\frac12\cdot\frac{5d^2}{2}=\frac{3d^2}{2}\tag{3}$$

which gives

$$d=\sqrt{\frac{2E_s}{3}}=\sqrt{2E_b}\tag{4}$$

Plugging (4) into (1) gives for the symbol error probablity

$$P_s=2.5\,Q\left(\sqrt{\frac{E_b}{N_0}}\right)\tag{5}$$

which, combined with (2), gives for the average bit error probability

$$E_b\approx\frac{5}{6}\,Q\left(\sqrt{\frac{E_b}{N_0}}\right)\tag{6}$$

---

Note that Eq. (6) for the bit error probability was derived under the assumption that the SNR is relatively high. This assumption is implicit in Eq. (1), where quadratic terms of the Q function were neglected. The exact expression for the symbol error probability is

$$P_s=2.5\,Q\left(\frac{d}{2\sigma}\right)-1.5\,Q^2\left(\frac{d}{2\sigma}\right)\tag{a}$$

For $d/2\sigma\rightarrow 0$ the expression (a) approaches $7/8$. This is a consequence of all symbols being detected with equal probability, and since only one of them is the correct symbol, the symbol error probability must be $7/8$.

The expression (2) for the bit error probability assumes that each symbol error causes only one bit error, which is true if Gray encoding is used and if it is assumed that symbol errors only occur between neighboring symbols, the latter being a reasonable assumption at relatively high SNRs. However, for $d/2\sigma\rightarrow 0$ any symbol will be confused with any other with equal probability. There is only one symbol (the correct one) where there are zero bit errors. There are $3$ symbols causing one bit error, $3$ symbols causing $2$ bit errors, and one symbol causing $3$ bit errors. Since for very bad SNRs all symbols are detected with equal probability, the average number of wrong bits per detected symbol is

$$\frac18\cdot 0+\frac{3}{8}\cdot 1+\frac{3}{8}\cdot 2+\frac18\cdot 3=\frac{3}{2}$$

And since there are $3$ bits per symbol, the average bit error rate for $d/2\sigma\rightarrow 0$ is

$$E_b=\frac{3}{2}\cdot\frac13=\frac12$$

as expected.