0
$\begingroup$

In many text books and research papers, I have seen that for the plot of Cramér-Rao Lower Bound vs Signal to Noise Ratio, the CRLB decreases with increasing SNR.

  • Is it possible that the CRLB remains constant for all SNR?
  • If so, what is the implication?
$\endgroup$
  • $\begingroup$ the CRLB could e.g. for high SNR approach some error floor due to e.g. noise-independent interference. $\endgroup$ – Maximilian Matthé Dec 19 '16 at 7:49
1
$\begingroup$

Prologue

The CRLB of an unbiased estimator $\hat \theta$ for the paramater $\theta$ is

$$ \text{var}(\hat \theta) \ge - \frac1 {\text E_x\left\{ \frac{\partial^2 (\ln f(x;\theta))}{\partial \theta^2} \right\}} $$

with $f(x;\theta)$ being the likelihood function of $\theta$ under the observation $x$. It is the inverse of the so-called Fisher information.

Answer

Is it possible that the CRLB remains constant for all SNR?

Yes. For that, the expectation (over all $x$, hence the index)

$$\text E_x\left\{ \frac{\partial^2 (\ln f(x;\theta))}{\partial \theta^2} \right\} $$

just needs to be constant.

The "easiest" explanation would be if noise just didn't have anything to do with the likelihood – ie. noise is just not actually observed by the estimator.

But that's kind of least exciting result – "well, if I add a lot of noise to something and then observe something completely unrelated, the noise has no effect" isn't that surprising.

A lot more complicated, is the option that the Fisher information is actually constant, although $f(x;\theta)$ actually depends on the noise.

The expectation operator $E_x$ is definable as

$$ \text E_x \{g(x)\} = \frac1{\mu (\xi)} \int\limits_{x\in\xi} g(x)\,d\mu\text, $$

i.e. as the integral over all possible $x$ divided by the measure $\mu$ of the set $\xi$ from which $x$ can be chosen.

Assuming smoothness of $x$ and compactness of $\xi\subset \mathbb R^1$ (just to make all the derivates easier to handle and to avoid cascading integrals), this specializes to:

$$\begin{align} \text E_x \{g(x)\} &= \frac1{x_{max}-x_{min}} \int\limits_{x = x_{min}}^{x_{max}} g(x)\,dx\\ \text E_x\left\{ \frac{\partial^2}{\partial \theta^2}{ (\ln f(x;\theta))} \right\} \cdot\left({x_{max}-x_{min}}\right) &= \int\limits_{x = x_{min}}^{x_{max}} \frac{\partial^2}{\partial \theta^2}{ (\ln f(x;\theta))} \,dx\\ &= \left[ \frac{\partial}{\partial \theta}{ (\ln f(x;\theta))} \right]_{x = x_{min}}^{x_{max}}\\ &= \left[ \frac1{f(x;\theta)} \frac{\partial f(x;\theta)}{\partial \theta} \right]_{x = x_{min}}^{x_{max}}\\ &= \frac1{f(x_{max};\theta)} \frac{\partial f(x;\theta)}{\partial \theta}_{x=x_{max}} - \frac1{f(x_{min};\theta)} \frac{\partial f(x;\theta)}{\partial \theta}_{x=x_{min}}\\ &= c,&c\text{ const.}\\ \end{align}$$

$\endgroup$
  • $\begingroup$ Thank you for the explanation. I do have a conceptual question which has been bugging me for a long long time. Could you please explain the following? (1) When computing the Fisher Information(FI), the mathematical expression contains $\theta$ and not $\hat{\theta}$. This means that FI is calculated based on the true, known value of the estimate. So, when implementing the expression of FI for any model, we plug in the true known value of theta and then compare it to the variance (MSE) of the estimate. Am I correct? $\endgroup$ – Ria George Dec 21 '16 at 3:08
  • $\begingroup$ (2) Say, I want to estimate the volume and area of a sphere. In layman's terms, the sphere contains data points which includes the actual information and some noise. The volume of the sphere is always fixed for a particular radius. However, when there is noise, the data points are far apart which may increase the volume and area of the sphere (container). We say that the entropy is high meaning the information content is high. So, Fisher Information can be high. As noise decreases, the sphere shrinks and hence its volume decreases. In this case would CRLB of the volume remain constant? $\endgroup$ – Ria George Dec 21 '16 at 3:12
  • $\begingroup$ This is really something you should ask in a proper question, not in comments. $\endgroup$ – Marcus Müller Dec 21 '16 at 9:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.