# Variance of expectation estimator

Given a set of $$N$$ iid samples, $$x_k$$, of random variable $$X$$ and the expectation estimator,

$$E[X] \approx \frac{1}{N} \sum_{k=1}^N x_k,$$

how do I find the variance of this estimator (not the variance of $$X$$) as a function of $$N$$?

I would only like to know the error in the estimate of $$E[X]$$ and what I would expect is that more samples (larger $$N$$) drives the variance down, so that

$$\lim_{N \to \infty} \bigg[\frac{1}{N} \sum_{k=1}^N x_k - E[X]\bigg] = 0.$$

Am I right about this? Perhaps variance is not the correct way to measure the error in the estimate? What I would like to understand is, how many $$N$$ is sufficient because there is a cost for every extra $$x_k$$ that is processed.

Without more knowledge on $$X$$, we cannot say a lot. However, if it is IID with mean $$\mu$$ and variance $$\sigma^2$$, the sample mean $$\hat{\mu} = \frac{1}{N}\left(\sum_{n=0}^{N-1}x_n\right)$$ is unbiased, since:

$$E[\hat{\mu}] = E\left(\frac{1}{N}\sum_{n=0}^{N-1}x_n\right) = \frac{1}{N}\sum_{n=0}^{N-1}E\left(x_n\right) = N\mu/N=\mu\,.$$

And its variance goes to zero when $$N$$ increases:

$$V[\hat{\mu}] = V\left(\frac{1}{N}\sum_{n=0}^{N-1}x_n\right) = \frac{1}{N^2}\sum_{n=0}^{N-1}V\left(x_n\right) = N\sigma^2/N^2=\sigma^2/N\,.$$

Thus, the expectation converges to the actual mean, and the variance of the estimator tends to zero as the number of samples grows. Under these definitions, the sample mean is a consistent estimator. Note that one could try to use other hypotheses: alternative norms, convergence in law, etc.

• Sorry, I corrected them. Indeed, the variance tends to 0 as the number of samples grows Jul 5, 2020 at 19:57
• I noticed that $E[x_n] = x_n$ because it is a constant, but you wrote $E[x_n] = \mu$. This seems incorrect. Jul 14, 2020 at 20:34
• We are talking about random variables. What is constant here? Jul 14, 2020 at 20:42