what is meant by biased and unbiased Autocovariance sequence estimate?

Question

In nonparametric spectral estimation method what is meant by biased and unbiased Autocovariance sequence estimate and why most commonly biased one is used? I have the answer to the second question from Peter Stoica (SPECTRAL ANALYSIS OF SIGNALS)but not getting what he wants to say.

Answer 1

In spectral estimation, one estimates "spectra" and other values. And one is interested in knowing whether the estimate $\hat{x}$ converge to the "true" but unknown value $x$. It is unknown, for instance because one has only access to a finite quantity $N$ of samples to compute with, a limited number of realization, because ergodicity is not assured, etc. So we want to know whether $\hat{x}_N\to x$ as $N\to \infty$. The convergence can be thought in different ways, and the most common is the mean square sense: does

$$\lim_{N\to \infty} E(|\hat{x}_N-x|^2)=0$$ where $E$ is the expectation? This can be really complicated to prove in practice. Thus, often people are satisfied when simpler conditions are met. And there are two necessary conditions for the above:

$$\lim_{N\to \infty} E(\hat{x}_N)=x$$

$$\lim_{N\to \infty} E(|\hat{x}_N|^2)=0$$

The first is the asymptotic unbiasedness, the second the vanishing variance. Together, they define consistent estimators. So, a biased autocorrelation estimate won't converge to the true autocorrelation. And derived estimators (like periodograms) are unlikely to converge as well.

Answer 2

Just for the sake of offering a different perspective, here's a basic explanation that tries to give some intuition without any theory.

Let's assume we have $N$ samples of a zero-mean random process $x(n), n=0...N-1$ and we want to estimate $R(d)=E(x(n)x^*(n-d))$. One obvious approach is to average $x(n)x^*(n-d)$ for all the data that we have:

$$\hat{R}(d)=\frac{1}{N-d}\sum_{n=d}^{N-1}{x(n)x^*(n-d)}$$

This is an unbiased estimate because it will converge to the true value as you use more and more data (larger $N$). But the estimates for larger lags $d$ will be less accurate than the estimates for smaller lags because we have less data available on which to base the estimates for the large lags.

Another approach is just to normalize the estimate by $1/N$ every time:

$$\hat{R}(d)=\frac{1}{N}\sum_{n=d}^{N-1}{x(n)x^*(n-d)}$$

This is a biased estimate because it does not converge to the true value, even when you use a lot of data. The estimates for larger lags $d$ are based on $N-d$ samples, but we normalize by $1/N$ regardless of the lag, so the estimates are a little too small. But the less accurate estimates (those with larger lags) are attenuated relative to the more accurate estimates (those with smaller lags), which can help. It is also often easier to compute: in fact, periodogram estimators (such as those that use the DFT) can be seen as using a biased autocorrelation.