0
$\begingroup$

In many single parameter estimation process average is the optimal. Can anyone tell under what condition average is optimal? For example is average optimal if the data is iid, Gaussian or something else?

$\endgroup$
2
$\begingroup$

You don't need to assume anything about the data in order for the average to be optimal (in a least squares sense) for certain estimation problems. As an example, assume you want to estimate a constant value $x$ and you are given $N$ noisy measurements:

$$y_n=x+e_n,\quad n=0,1,\ldots,N-1\tag{1}$$

where $e_n$ is the error of the $n^{th}$ measurement. If our goal is to minimize the mean squared error

$$\frac{1}{N}\sum_{n=0}^{N-1}e_n^2=\frac{1}{N}\sum_{n=0}^{N-1}(y_n-x)^2\tag{2}$$

then the optimal estimate $\hat{x}$ is obtained by minimizing (2). Taking the derivative of the right-hand side of (2) with respect to $x$ and setting it to zero gives

$$\frac{2}{N}\sum_{n=0}^{N-1}(y_n-\hat{x})=0\tag{3}$$

From (3) the optimum estimate is obtained as

$$\hat{x}=\frac{1}{N}\sum_{n=0}^{N-1}y_n\tag{4}$$

which is simply the average of all observations.

$\endgroup$
  • $\begingroup$ This seems not very true, or at least true only for zero mean type of noises. You have to assume certain noise statistics in order for the simple average to be optimal estimator of any quantity within noise. As an example, consider the noise sequence $e_n$ to be a uniform noise between [1,3]. Then would you still argue that average of the observations will be the optimal? $\endgroup$ – Fat32 Jul 1 '15 at 17:23
  • $\begingroup$ @Fat32: Yes, I would still argue that it is optimal in the least squares sense. You have to distinguish between "optimal" and "good". $\endgroup$ – Matt L. Jul 1 '15 at 17:30
  • $\begingroup$ Can I assume mean is optimal in least square for a zero mean noise, no more very specific case? $\endgroup$ – Creator Jul 1 '15 at 18:20
  • $\begingroup$ @Creator: Note that I didn't make any assumption on the mean of the noise in my answer. Of course, for noise with a non-zero mean value the estimate will be biased, but still optimal according to the least squares criterion. This result remains true if you replace least squares by minimum mean square error, i.e. if you assume that the noise is a random variable with a given PDF. $\endgroup$ – Matt L. Jul 1 '15 at 20:07
  • $\begingroup$ I understood that it is independent of the noise PDF. Is it correct? $\endgroup$ – Creator Jul 1 '15 at 20:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.