I am new in learning deriving BER expression. I am familiar for the case of BPSK in AWGN but cannot understand how the BER expression in Eq(12) in this paper, An Enhanced Spectral Efficiency Chaos-Based Symbolic Dynamics Transceiver Design download link
tutorial presents the BER expression for M-QAM in AWGN.
Do you have any reference on how to analytically derive the BER for QAM and Rayleigh in AWGN for this case? How can I derive?