In Fundamentals Of Statistical Signal Processing: Estimation Theory page 19, Kay mentions a biased mean square error estimator for $\mu$ where the samples $x\sim\text{N}(\mu, \sigma^2)$.
The suggested estimator is $$\hat{\mu}=\frac{a}{N}\sum_{i=0}^{N}x_i$$ where optimal $a$ is given by $$a=\frac{\mu^2}{\mu^2 + \sigma^2/N}$$
He also mentions that those kind of estimators are not practical due to the dependency of the parameter $a$ on the unknown parameters $\mu$ and $\sigma^2$.
Now, I have tried to implement this estimator iteratively, using python/numpy and it provided me with some decent results.
nTests = 100000
unbiasError = np.empty(nTests)
biasError = np.empty(nTests)
for test in range(nTests):
x = np.random.normal(loc=1, scale=10, size=1000)
#init
a = 1
aHist = 0.5
flag = 1
while (np.abs(a - aHist) > 0.00001):
if flag == 0:
meanEst = x.mean() * a
else:
meanEst = x.mean()
flag = 0
varEst = ((x - meanEst)**2).sum() / (x.size - 1)
aHist = a
a = (meanEst**2) / (meanEst**2 + varEst / x.size)
biasError[test] = meanEst - 1
unbiasError[test] = x.mean() - 1
print("mean error of the biased estimator: {:.2f}\nmean error of the unbiased estimator: {:.2f}\n".format(
np.abs(biasError).mean(), np.abs(unbiasError).mean()))
mean error of the biased estimator: 1.23
mean error of the unbiased estimator: 2.52
I have to admit that it works better on low SNR (For $\sigma^2=10$ the unbiased estimator performs better). Therefore, it is not clear to me if it is completely not practical to use such estimators. Are there clear criteria for the stability and performance of such methods? Is there any known better implementation for such estimator?
.
