Considering an estimation problem of estimating a scalar deterministic parameter $a$ from the observations $y$ which are corrupted by randomvariable $w$. The observations are $y[n] = a + w[n]$

Least Squares estimator can be used to estimate $a$ when $w$ is a White Gaussian Random Variable. This estimation method is known to be optimal. Why?

What if $w$ is from Poisson Distribtuion or some other non-gaussian, then would the estimator for $a$ be better or worse than the one found using $w$ as Gaussian r.v?


So Least squares estimator is as it literally - the estimator which brings the mean square error to minimum. In the case of Gaussian white noise it has a simple and analytic solution. I recommend you develop it yourself, if your'e comfortable with matrices calculus it is not that hard.

You can generate tons of estimators by defining different cost functions. Other popular cost functions are l1-norm ($\sum |x_i - y_i|$), likelihood ($P(y|x)$).

It's hard to define "better" if the case isn't the same, For example, does the case of very high variance poisson process is the same as the same variance gaussian process? I believe it is not, and in addition one is non negative integer while the other is real (and you can use it for you advantage).

| improve this answer | |
  • $\begingroup$ THank you for your reply, but it is not clear how you define "better" estimator when we have several estimators to estimate the unknown parameter. $\endgroup$ – SKM Jul 28 '17 at 5:17
  • $\begingroup$ sorry for the late response but here I am. So estimation "optimality" depends on the problem: do you have the parameters distribution and the distribution of the measurements given the parameter? is there even a distribution for the parameter? maybe the data is finite and you don't need the worry about distributions? I'm sorry I'm spamming you but estimation is a pretty large field, and I don't think I can answer here with a comment :) $\endgroup$ – Cherny Jul 31 '17 at 7:35
  • $\begingroup$ Thanks for your comment. There is no distribution for the parameter a, it is a deterministic value and I have obtained it randomly from a Gaussian distribution. I am checking which noise distribution gives the estimate of the parameter that is the "best" or closest to the known value. This is the reason I ask if there is any theory which says how to determine which estimator is the best. $\endgroup$ – SKM Aug 3 '17 at 21:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.