Let's suposse I have a vector of elements $x(n) = \{x(0), x(2), \cdots ,x(N-1)\}$ from a random process X of mean $\mu_x$ and variance $\sigma_x^2$. I want to see if I can estimate the mean and variance of the random process with my finite vector.

So, we say an estimator is unbiased if the mean of the estimator is the same as the mean of the random process. For example, the sample average is:

$$\hat{\mu}_x = \dfrac{1}{N} \sum_{n = 0}^{N-1} x(n)$$

Taking average in each member:

$$E[\hat{\mu}_x] = \dfrac{1}{N} \sum_{n = 0}^{N-1} E[x(n)] = \dfrac{1}{N} \sum_{n = 0}^{N-1} \mu_x = \mu_x$$

Thus, the sample mean is unbiased. My problem comes with the mean of the sample variance. The definition of the sample variance is the following:

$$\widehat{\sigma_x^2} = \dfrac{1}{N} \sum_{n = 0}^{N-1} (x(n) - \hat{\mu}_x)^2 $$

I have to prove that:

$$E\left[\widehat{\sigma_x^2}\right] = \sigma_x^2 \left(1 - \dfrac{1}{N} \right) $$

I have tried to expand the square parenthesis:

$$\widehat{\sigma_x^2} = \dfrac{1}{N} \sum_{n = 0}^{N-1} E[x^2(n)] + E[\hat{\mu}_x^2] - 2 E[x(n) \hat{\mu}_x]$$

Nevertheless, when I have to calculate $E[\hat{\mu}_x^2]$ (I suppose it's this way):

$$E[\hat{\mu}_x^2] = \dfrac{1}{N^2} E\left[ \left( \sum_{n = 0}^{N-1}x(n) \right)^2 \right]$$

I don't know how to do it. Moreover, I don't know if $E[x(n) \hat{\mu}_x] = E[x(n)] E[\hat{\mu}_x]$ (I think that's not correct in general), and I have tried to solve it in another way, but anything comes to my mind.

I hope someone can help me. Thank you for your responses.

  • 2
    $\begingroup$ 1) I'm voting to close this question as off-topic because at its core, this is a question about mathematics rather than DSP. It has higher chances of finding an answer in boards such as Math Overflow or Mathematics.SE. 2) That first expression about the mean and the conclusion that it is unbiased seems a bit rushed (?). You might find this helpful. $\endgroup$ – A_A Mar 7 '19 at 7:02
  • $\begingroup$ I'm asking here the question because in MathSE nobody gave me an answer $\endgroup$ – Josemi Mar 9 '19 at 10:41

The key to prove $$ E\left[\widehat{\sigma_x^2}\right] = \sigma_x^2 \left(1 - \dfrac{1}{N} \right) $$ is to add and subtract $\mu_x$ in the expression for the sample variance before working out the square. A detailed proof is available on the Wikipedia page for Bessel's correction.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.