Bounding Detection and Estimation by SNR in Gaussian Channel

Question

Assume the following problem: A deterministic signal $X$ whose magnitude is known to satisfy $0 \leq \Vert X \Vert_2 < \Delta$ for some known constant $\Delta$ is transmitted through a Gaussian Channel with known standard deviation $\sigma$.

It is known that the ideal observer performance on the detection problem (detectability) or ROC-AUC is upper-bounded by the SNR which in this case is at most $\left(\Delta \over \sigma\right)^2$.

My question: What does the SNR tell us about the error in estimating $X$? By Cramér-Rao, it's evident that the variance of an unbiased estimator will be at least $\sigma^2$, but how can we use the fact that the SNR is bounded within a certain interval? It should improve the estimation by introducing bias. I want to provide an exact equation relating the SNR to the error of the estimator.

Answer 1

Going directly for the variance is hard, because we don't know the pmf/pdf/cdf/moment-generating function/characterstic function/… of your $X$.

What we can do, instead, is going through entropy, and in the end look at the worst case for that. Let's assume $X$ is continuously-valued and has a differentiable cumulative density function $F_X$ and hence a probability density function $f_X$.

1. The (differential) entropy $h(X)$ (in bits) of $X$ is bounded to $\log_2 \Delta$. \begin{align} h(X):=\ & \int-\log_2(f_X(x) )f(x)\,\mathrm dx\\ =\ &\int\log_2\left(\frac1{f_X(x)}\right) \mathrm dF_X(x)\\ &\text{and we make that log well-defined by}\\ &\text{applying it only where the pdf takes non-}\\ &\text{zero values (outside, the entropy contri-}\\ &\text{buted is 0, anyways) and is bounded (we}\\ &\text{required $X$ to be continously distributed)}\\ =\ &\int_0^{\Delta}\log_2\left(\frac1{f_X(x)}\right) \mathrm dF_X(x)\\ &\text{Now, we realize $\log$ is always a concave}\\ &\text{functions, and thus, }\textbf{Jensen's Inequality}\\ &\text{applies (yeah, measurability is given by $f$}\\ &\text{being the derivative of $F$).}\\ \le\ &\log_2\left( \int_0^\Delta \frac1{f_X(x)}\mathrm dF(x) \right) \tag{☮}\label{le} \\ =\ &\log_2\left( \int_0^\Delta \frac1{f_X(x)} f(x) \mathrm dx \right)\\ =\ &\log_2\left( \int_0^\Delta 1 \mathrm dx \right)\\ =\ &\log_2\Delta \end{align}
2. Since noise is independent to signal, $h(Y)=h(X+N)=h(X) + h(N)$, with $h(N)=\frac 12\log_2(2\pi e\sigma^2)+\frac12$: $$h_\max(Y) = \frac 12\log_2(2\pi e\sigma^2\Delta^2) +\frac12 .$$

At this point, it's usually good to actually stop – knowing the variance tells us less about our estimator than knowing its entropy - but if we want to make conclusions on the optimal estimator variance:

3. The weak inequality $\eqref{le}$ becomes an equality for $1/f_X$ being a constant (without proof, but easy to test), so the distribution with the highest possible variance for a random variable $X$ with finite support $[0,\Delta]$ is the uniform distribution $f_X(x) = 1/\Delta, x\in[0,\Delta], f_X(x) = 0\text{ else}$. This makes the maximum likelihood estimator for this worst case easy to derive – and you should be able to read the variance from that.