# Bounding Detection and Estimation by SNR in Gaussian Channel

Assume the following problem: A deterministic signal $$X$$ whose magnitude is known to satisfy $$0 \leq \Vert X \Vert_2 < \Delta$$ for some known constant $$\Delta$$ is transmitted through a Gaussian Channel with known standard deviation $$\sigma$$.

It is known that the ideal observer performance on the detection problem (detectability) or ROC-AUC is upper-bounded by the SNR which in this case is at most $$\left(\Delta \over \sigma\right)^2$$.

My question: What does the SNR tell us about the error in estimating $$X$$? By Cramér-Rao, it's evident that the variance of an unbiased estimator will be at least $$\sigma^2$$, but how can we use the fact that the SNR is bounded within a certain interval? It should improve the estimation by introducing bias. I want to provide an exact equation relating the SNR to the error of the estimator.

• While you're right, Cramér-Rao only holds for unbiased estimation, adding a bias does not inherently reduce variance! Now, I do agree, there's probably something to be won here. Commented Aug 8, 2022 at 15:09
• are you really interested in "what can we read from a bounded SNR?" (because, that's just going to bound the variance), or in "what can we read if we know the support of $X$ and the PDF of the noise?". The latter can give you a smaller variance, but it's not what you ask for in the letter. Commented Aug 8, 2022 at 15:36
• your SNR formula needs to be squared, by the way! Commented Aug 8, 2022 at 15:44
• @MarcusMüller To be honest with you, what I am hoping for is a bound of the type: "If your hypothesis test/ detection AUC is at most A, then your MSE can be as low as B, where B is some function of the SNR". Also good catch on the square :)
– Sami
Commented Aug 8, 2022 at 18:55

Going directly for the variance is hard, because we don't know the pmf/pdf/cdf/moment-generating function/characterstic function/… of your $$X$$.

What we can do, instead, is going through entropy, and in the end look at the worst case for that. Let's assume $$X$$ is continuously-valued and has a differentiable cumulative density function $$F_X$$ and hence a probability density function $$f_X$$.

1. The (differential) entropy $$h(X)$$ (in bits) of $$X$$ is bounded to $$\log_2 \Delta$$. \begin{align} h(X):=\ & \int-\log_2(f_X(x) )f(x)\,\mathrm dx\\ =\ &\int\log_2\left(\frac1{f_X(x)}\right) \mathrm dF_X(x)\\ &\text{and we make that log well-defined by}\\ &\text{applying it only where the pdf takes non-}\\ &\text{zero values (outside, the entropy contri-}\\ &\text{buted is 0, anyways) and is bounded (we}\\ &\text{required X to be continously distributed)}\\ =\ &\int_0^{\Delta}\log_2\left(\frac1{f_X(x)}\right) \mathrm dF_X(x)\\ &\text{Now, we realize \log is always a concave}\\ &\text{functions, and thus, }\textbf{Jensen's Inequality}\\ &\text{applies (yeah, measurability is given by f}\\ &\text{being the derivative of F).}\\ \le\ &\log_2\left( \int_0^\Delta \frac1{f_X(x)}\mathrm dF(x) \right) \tag{☮}\label{le} \\ =\ &\log_2\left( \int_0^\Delta \frac1{f_X(x)} f(x) \mathrm dx \right)\\ =\ &\log_2\left( \int_0^\Delta 1 \mathrm dx \right)\\ =\ &\log_2\Delta \end{align}
2. Since noise is independent to signal, $$h(Y)=h(X+N)=h(X) + h(N)$$, with $$h(N)=\frac 12\log_2(2\pi e\sigma^2)+\frac12$$: $$h_\max(Y) = \frac 12\log_2(2\pi e\sigma^2\Delta^2) +\frac12 .$$

At this point, it's usually good to actually stop – knowing the variance tells us less about our estimator than knowing its entropy - but if we want to make conclusions on the optimal estimator variance:

1. The weak inequality $$\eqref{le}$$ becomes an equality for $$1/f_X$$ being a constant (without proof, but easy to test), so the distribution with the highest possible variance for a random variable $$X$$ with finite support $$[0,\Delta]$$ is the uniform distribution $$f_X(x) = 1/\Delta, x\in[0,\Delta], f_X(x) = 0\text{ else}$$. This makes the maximum likelihood estimator for this worst case easy to derive – and you should be able to read the variance from that.
• Thanks! I'm not hell-bent on the variance, I'm happy with some other estimate of the error for the estimation problem (e.g. (M)MSE). My main question is (cf. comment above) whether a bound on detection (which depends only on SNR) implies a bound on estimation which involves the SNR.
– Sami
Commented Aug 8, 2022 at 19:00
• Also, regarding the parameterisation of the problem: $X$ is assumed to be a Dirac impulse which we may assume to be parameterised by some other variable $\alpha$, but all that does is to move around the mean of the resulting Gaussian. That said, it may be of interest to consider what an error bound on estimating $\alpha$ may be as opposed to $X$.
– Sami
Commented Aug 8, 2022 at 19:02
• That would only require you to calculate the transformed variable's pdf; this is done using the Jacobi determinant of the transformation and the raw variable. Commented Aug 8, 2022 at 19:11
• One thing seems a bit counter-intuitive, although the derivation makes sense: You would expect that conditioning (i.e. the fact that we know that $\Delta^2 \over \sigma^2$ is bounded) should decrease uncertainty, whereas the noise in the channel should increase it. The way it looks, the higher the magnitude of the signal ($\Delta^2$), the higher the uncertainty becomes?
– Sami
Commented Aug 9, 2022 at 10:48
• yeah, sure! The more the signal can vary, the less you know about it! Commented Aug 9, 2022 at 13:47