Choosing the scaling parameters for a modem implementation

Question

I have succesfully modeled a modem using floating-point data and with the following scheme, where the SRRC filter has gain G=6dB.

Modulator -> SRRC(UPx4) -> D/A -> Channel -> A/D -> AGC -> SRRC(DOWNx2) -> Remaining receiver blocks

The modulator outputs full-scale (-1 to +1) I/Q values and at the receiver I'm getting a signal whose power is 1/4=0.25W as expected, the digital AGC uses that value as target, and after the other filter I do get back a more-or-less unitary-energy signal. The reason to keep the ADC there is to provide the other blocks (e.g. Gardner synchronizer) with a signal whose amplitude is almost unitary.

Now, I'm not sure this is the best way to structure the TX/RX chain but I'm pretty sure this set-up won't fly that well when converted to fixed-point arithmetic: what I'm afraid of is that the AGC/SRRC combination as devised may cause the signal to clip/overflow and thus degrade the receiver performance. Is there a better and perhaps wiser way of designing the receiver chain? Use normalized filters? Use a different target value for the AGC?

The bit-growth caused by the two SRRC filters doesn't worry me, the plan is to compute the result with higher precision and then round/truncate it to 16-bit.

I'm also considering to lower a little the AGC target level, instead of targeting 1/4 of the quantized power scale the reference could be moved to a smaller value (e.g. 1.25) and reduce the chances of clipping.

I'm pretty new to all this DSP stuff so please bear with me, I'm trying to learn as much as possible while also trying to get a product out of the door :)

Answer 1

The optimum solution for where to set the Automatic Gain Control (AGC) relative to the full scale of the ADC typically allows for some clipping given the goal is to minimize the total noise degradation consisting of both clipping noise and quantization noise. As we lower the signal level at the input to the ADC, quantization noise increases relative to the signal level while clipping noise decreases, and the opposite occurs as we increase the signal level. I have created the following chart which demonstrates this showing the SNR vs number of bits for Gaussian distributed waveforms. A Gaussian distribution is applicable to most modern waveforms, but the methodology can be applied using the specific expected distribution for a waveform of interest.

The chart is applicable to both real sampling with a single ADC as well as complex sampling with a dual ADC. For real sampling, full scale is where a Sine wave would start to clip and would be -3 dB on the horizontal axis. For complex sampling a similar "tone" would have constant magnitude as $Ae^{j\omega t}$ and would be 0 dB on the horizontal axis when at full scale where clipping starts to occur. The horizontal axis is the set point of the AGC relative to full scale and the vertical axis is the resulting total noise relative to the power in the signal (so negative SNR), depending on the Effective Number of Bits (ENOB) for the converter. For example if we had an ENOB of 10 bits, then for a real waveform the optimum AGC set point would be at approximately -13 dB on the horizontal axis, and the noise at this set point is approximately the best it can be at -51 dB. For a real signal, this is 10 dB below the that level where a sine wave would clip. So if in our given system we knew a sine wave would clip at +10 dBm (for example), we should AGC to 0 dBm and maximize our dynamic range and SNR.

I created this chart by combining the SNR due to quantization noise (using the well established formula $SNR = 6.02 \text { dB/bit} + 1.76$ dB with the predicted SNR due to clipping alone for a Gaussian distributed waveform. This SNR assumes the full Nyquist zone is being used, otherwise subsequent filtering will offer further improvement. Thankfully the detailed math for the case of clipping (from the area under the tail of a Gaussian) has already been worked out by Walt Kester and Rob Reeder in their article about Noise Power Ratio referenced in my graphic below.

As noted, as the distribution deviates from a Gaussian, the optimum set point will vary, and I recommend reviewing the actual distribution assuming equiprobable data (unless known otherwise) and if it deviates signifcantly from a Guassian (higher or lower peak to average power), then the curve shown above "ADC SNR due to Clippling" can be created (even through simulation) to combine as I have done with the quantization noise curves for any specific application. Further the spectral density of the clipping noise can be reviewed if subsequent filtering is to be done to understand how much of the clipping noise may be subsequently rejected by the filtering. The main point is that it is often a mistake to design the receiver with an AGC level such that there is no clipping; which means being far to the left of the "ADC SNR due Clipping" curve such that the quantization noise contribution increases more than would have been realized at a minimum operating point where some clipping is allowed.