Signal Processing Stack Exchange is a question and answer site for practitioners of the art and science of signal, image and video processing. It only takes a minute to sign up.
Sign up to join this community
I have a gaussian signal $x$ with zero mean and unit variance. I am quantizing it by using a uniform quantizer of step size $q$ calculated by
$$q = \frac{x_\mathrm{max} - x_\mathrm{min}}{2^{M_\mathrm{ENOB}}},$$
where $M_\mathrm{ENOB}$ is the effective number of bits. Is this the correct way of quantizing a Gaussian signal?
The problem with quantizing Gaussian distributed signals (like the real/imaginary part of an OFDM signal) is that they can take any value in theory. It is thus necessary to clip such signals at threshold $C$ prior to quantization.
Low $C$ increases the distortion noise in this process, while large $C$ will lead to strong quantization noise. It can be shown, that there is an optimum value for $C$ that depends on the variance $\sigma^2$ of the signal and the quantizer resolution $M$. For details see [1,2]. After $C$ has been determined, the step size $q$ is calculated by $$ q=\frac{2C}{2^M} $$
By the way, I don't think it makes sense to use the ENOB (of a DAC) for calculating the step size. The ENOB is a kind of signal-to-noise ratio combining quantization noise and other effects like frequency selectivity and spurious frequencies of the DAC device. The number of input bits is still the nominal number of bits $M$.
Example: let $\sigma^2=1$ and $M=8$. Then with eq. (7) from [2] we can calculate the SNR $\gamma$ after clipping and quantization as a function of the clipping ratio $\zeta=C^2/\sigma^2$. The SNR should be maximized. I don't know a closed-form expression for the optimum $\zeta_\mathrm{opt}$ yielding $\gamma_\mathrm{max}$ but we can determine it numerically. For $M=8$ and with Fig. 2 from [2] we have $\zeta_\mathrm{opt} \approx 11.87\,\text{dB} \approx 15.38$. Furthermore, $C_\mathrm{opt}=\sqrt{\zeta_\mathrm{opt}\sigma^2}$ and with $\sigma^2=1$ we finally get $C_\mathrm{opt}\approx3.92$.
Remark: as you tagged this question digital-communictions you might be dealing with complex signals. In that case be careful with the definition of $\sigma^2$. In my answer it is the variance of a real-valued, Gaussian distributed signal. If the signal in question is complex-valued with independent and identically Gaussian distributed real and imaginary part, it has variance $2\sigma^2$, where $\sigma^2$ is the variance of the real or imaginary part.
References
[1] D. J. G. Mestdagh, P. Spruyt, and B. Biran, “Analysis of clipping effect in DMT-based ADSL systems,” in Communications, 1994. ICC’94, SUPERCOMM/ICC'94, Conference Record,'Serving Humanity Through Communications.'IEEE International Conference on, 1994, pp. 293–300.
[2] M. Bernhard, D. Rörich, T. Handte, and J. Speidel, “Analytical and numerical studies of quantization effects in coherent optical OFDM transmission with 100 Gbit/s and beyond,” in ITG Symposium on Photonic Networks, 2012, pp. 34–40. Online
