Demonstrate the mean of the sample variance

Question

Let's suposse I have a vector of elements $x(n) = \{x(0), x(2), \cdots ,x(N-1)\}$ from a random process X of mean $\mu_x$ and variance $\sigma_x^2$. I want to see if I can estimate the mean and variance of the random process with my finite vector.

So, we say an estimator is unbiased if the mean of the estimator is the same as the mean of the random process. For example, the sample average is:

$$\hat{\mu}_x = \dfrac{1}{N} \sum_{n = 0}^{N-1} x(n)$$

Taking average in each member:

$$E[\hat{\mu}_x] = \dfrac{1}{N} \sum_{n = 0}^{N-1} E[x(n)] = \dfrac{1}{N} \sum_{n = 0}^{N-1} \mu_x = \mu_x$$

Thus, the sample mean is unbiased. My problem comes with the mean of the sample variance. The definition of the sample variance is the following:

$$\widehat{\sigma_x^2} = \dfrac{1}{N} \sum_{n = 0}^{N-1} (x(n) - \hat{\mu}_x)^2 $$

I have to prove that:

$$E\left[\widehat{\sigma_x^2}\right] = \sigma_x^2 \left(1 - \dfrac{1}{N} \right) $$

I have tried to expand the square parenthesis:

$$\widehat{\sigma_x^2} = \dfrac{1}{N} \sum_{n = 0}^{N-1} E[x^2(n)] + E[\hat{\mu}_x^2] - 2 E[x(n) \hat{\mu}_x]$$

Nevertheless, when I have to calculate $E[\hat{\mu}_x^2]$ (I suppose it's this way):

$$E[\hat{\mu}_x^2] = \dfrac{1}{N^2} E\left[ \left( \sum_{n = 0}^{N-1}x(n) \right)^2 \right]$$

I don't know how to do it. Moreover, I don't know if $E[x(n) \hat{\mu}_x] = E[x(n)] E[\hat{\mu}_x]$ (I think that's not correct in general), and I have tried to solve it in another way, but anything comes to my mind.

I hope someone can help me. Thank you for your responses.

Answer 1

The key to prove $$ E\left[\widehat{\sigma_x^2}\right] = \sigma_x^2 \left(1 - \dfrac{1}{N} \right) $$ is to add and subtract $\mu_x$ in the expression for the sample variance before working out the square. A detailed proof is available on the Wikipedia page for Bessel's correction.