# Negative bias in sample auto-correlation of uncorrelated time-series

I am using the following definition for the sample auto-correlation:

$$AC_x(l)=∑_{i=1}^{N-l}(x_i-μ)(x_{i+l}-μ)$$ ; $$μ=\frac{1}{N}∑_{i=1}^Nx_i$$

I have noticed that this definition introduce a systematic negative bias to the results due to the subtraction of the sample mean. I calculated the auto-correlation of a white noise sample and after repeating that for thousands of realizations and averaging them in each time lag, I got that bias (attached are my code).

I was really surprise at first, but then, I calculated $$E(AC_x(l))$$ ($$E$$ for expectancy value) by hand and got: $$E(AC_x(l))=-(1-\frac{l}{N})E(x_i^2)$$ $$^1$$ (for $$l>0$$). This is the exact pattern I got in my simulations.

So, my question is how should I interpret that? And which definition I should use for auto-correlation in order to get the intuitive result of $$\delta(l)$$?

def autocorrelation(x):
x_mean = np.mean(x)
x_var = np.sum((x-x_mean)**2)
corrs = []
for i in range(len(x)):
test_x1 = x[:(len(x)-i)]
test_x2 = x[i:]
corrs.append(np.sum((test_x1 - x_mean) * (test_x2 - x_mean)) / x_var)
return np.array(corrs)

s = np.random.normal(size=(10000, 10))
corrs = np.array([autocorrelation(s[i]) for i in range(s.shape[0])])

plt.figure(figsize=(4*3, 3*3))
plt.errorbar(np.arange(1, corrs.shape[1]), np.mean(corrs[:, 1:], axis=0), st.sem(corrs[:, 1:], axis=0))
plt.xlabel('time lag')
plt.ylabel('correlation')
plt.xticks(ticks=np.arange(1, corrs.shape[1]))
plt.show()


$$^1$$ A full proof of the bias expression:

\begin{align} E(AC_l (x))&=E(\sum_{i=1}^{N-l}(x_i-μ)(x_{i+l}-μ))\\ &=\sum_{i=1}^{N-l}E((x_i-μ)(x_{i+l}-μ))\\ &=\sum_{i=1}^{N-l}E(x_i x_{i+l}-x_i μ-x_{i+l} μ+μ^2 )\\ &=\sum_{i=1}^{N-l}E(x_i x_{i+l} )-E(x_i μ)-E(x_{i+l} μ)+E(μ^2 )\\ &=\sum_{i=1}^{N-l}E(x_i x_{i+l})\\ &\phantom{=\sum_{i=1}^{N-l}}-E\left(x_i\frac {x_1+⋯+x_N }{N}\right)\\ &\phantom{=\sum_{i=1}^{N-l}}-E\left(x_{i+l}\frac {x_1+⋯+x_N }{N}\right)\\ &\phantom{=\sum_{i=1}^{N-l}}+E\left(\left(\frac {x_1+⋯+x_N }{N}\right)^2\right)\tag{a} \end{align}

Now, for $$i\ne j$$: $$E(x_i x_j )=E(x_i )E(x_j )=0$$, because they are independent and each with 0 mean, therefore a lot of mixed elements will fall off, and we will stay with:

\begin{align} &=\sum_{i=1}^{N-l}\left(-E(\frac{x_i^2}{N})-E(\frac{x_{i+l}^2}{N})+E(\frac{x_1^2+⋯+x_N^2}{N^2})\right)\tag{b}\\ &=\sum_{i=1}^{N-l}\left(-\frac{E(x_i^2 )}{N}-\frac{E(x_{i+l}^2)}{N}+\frac{E(x_1^2)+⋯+E(x_N^2)}{N^2}\right)\tag{c} \end{align}

$$x_i$$ are identically distributed, therefore:

\begin{align} &=\sum_{i=1}^{N-l}\frac{-NE(x_i^2 )-NE(x_i^2 )+NE(x_i^2 )}{N^2}\\ &=\sum_{i=1}^{N-l}-\frac{E(x_i^2 )}{N}\tag{*}\\ &=-\frac{N-l}{N} E(x_i^2) \\ &=-(1-\frac{l}{N})E(x_i^2) \end{align}

And this is exactly what the simulation shows for N=10

• if your noise is white, then $\mu\equiv 0$. Otherwise, the noise wasn't white. Attention, you'd need an infinitely large $N$ to really measure that, though the error gets smaller proportionally to $N$. – Marcus Müller Jan 1 '20 at 11:30
• @MarcusMüller Thanks a lot for your response. 1. I was asking on the interpretation of the auto-correlation of a finite sample of some random process (white noise, for example). At first, I was expecting to get a delta function, but actually I got a non-zero bias (that can be mathematically expressed, as I showed). 2. This negative bias does not get smaller as N get larger, it's actually increases (in absolute term) and converges to -1. I would be very glad to hear your interpretation for finite sample auto-correlation, or another valid way to calculate it. – Tomer Milo Jan 1 '20 at 13:11
• :) I'd like to, but really, the expectation of the autocorrelation of a white series is 0, per definition of "white" for any non-zero lag $l$ (there, it's its Variance). Could you maybe add the calculation that led to your expectation value to your question? – Marcus Müller Jan 1 '20 at 13:42
• @MarcusMüller, I attached the mathematical "proof" above in the main text cause it was too long for comment. Thanks a lot! – Tomer Milo Jan 1 '20 at 14:55
• beautified your equations a bit :) Let me have a look at it, might take a few minutes, though. – Marcus Müller Jan 1 '20 at 15:03