Implication of Cramer Rao Lower bound and relation with SNR : non-technical overview

Question

I need some clarifications on CRLB which has been bugging me for quite some time.

• Clarification: When computing the Fisher Information(FI), the mathematical expression contains $\theta$ and not $\hat{\theta}$ wnere $\theta$ is the true known parameter that we wish to estimate and $\hat{\theta}$ is its estimate. This means that FI is calculated based on the true, known value of the estimate. So, when implementing the expression of FI for any model, we plug in the true known value of theta and then compare it to the variance (MSE) of the estimate. Am I correct?

• Confusion: The Mean Square error (MSE) between the true and the estimated volume may decrease with increasing Signal to Noise Ratio. CRLB is a lower bound for MSE. In that case, would CRLB decrease with increasing SNR?

Answer 1

This means that FI is calculated based on the true, known value of the estimate.

No! The Fisher estimation is based on the expectation of the likelihood function. The likelihood is just a normal probability – just that the role of the "original" variable and observation have been reversed.

I don't understand your example – spheres don't have a CRLB, the problem of estimating a certain parameter has.

Maybe you're confusing the "set of all estimators for $\theta$" (which has a CRLB) with "a specific estimator" (which simply has a variance) or a with "a specific estimate" (which has an error). That is, the Fisher info is a function of the "original" $\theta$.

So, Can Fisher Information be high and does it vary with the noise?

So imagine a binary source: there's only two possible $\theta$, and they are sent through a memoryless AWGN channel – that is, you observe $x$, and it's either caused by $\theta=0$ or $\theta=1$, and figuring that out is your estimator's job (which gives you $\hat\theta$).

The likeliness function $f(x;\theta)$ hence gives you an idea of how likely an outcome $x$ is explained by the original effect $\theta$ was.

Obviously, if there's very little noise, that function will have a very "sharp" shape – there's little chance a 0 sent will be interpreted as 1 in the receiver.

Increasing the noise power makes it possible that this actually happens – the likelihood changes, and it becomes much more probable that the other $\theta$ value is the reason for what you're seeing.