# Estimating quantification and sampling error

I have a signal sampled from $$\mathbb{R}^2$$ at a frequency of $$2\text{Hz}$$. It is quantized from $$\mathbb{R}^2$$ to $$\mathbb{N}^2$$, seemingly using the round operator.

How can I estimate the error variance from this setup ? I've found models for univariate signals but they don't factor in the sampling error, and I'm not sure how I can simply generalize the quantization error from $$\mathbb{R}$$ to $$\mathbb{R}^2$$.

• What does "sampled from R^2" mean? Mar 29, 2022 at 9:27
• @RichardLyons each sample of the signal is a 2-dimensional vector Mar 29, 2022 at 10:02
• @hdubois you say the values seem to be rounded, so it does sound like you at least have an idea of the underlying 2D probability density of the original signal, right? Do you know anything about the temporal behaviour of the signal, as well? (autocorrelation/PSD?) Without having a hypothesis of how the original signal was distributed, we can't form a hypothesis on the error. For all we know, you could be in immensive luck and the original data was arbitray close to points in $\mathbb N^2$ to begin with. Mar 29, 2022 at 10:50
• The answers at this post may interest you: dsp.stackexchange.com/questions/39025/… Mar 29, 2022 at 15:01

Let's call your signal $$\vec{x}$$ and the quantized signal $$\vec{x_q}$$ . If you define the error simply as the mean square error between those, i.e.
$$E = \left\langle\left\lvert\vec{x_q}-\vec{x_{}}\right\rvert^2\right\rangle$$
the error will (in most cases) be simply twice the error for the one dimensional case. Assuming rounding and a quantization step of $$\Delta$$ the 1-D error would (in most cases) be
$$E = \frac{1}{12} \Delta^2$$
"In most cases" means the signals are significantly larger than the quantization steps, the two dimensions are uncorrelated and the probability density function of the original signal is reasonably continuous. In this case the quantization error is uniformly distributed on [$$-\Delta/2,+\Delta/2]$$.