I am using the following definition for the sample auto-correlation:

$AC_x(l)=∑_{i=1}^{N-l}(x_i-μ)(x_{i+l}-μ)$ ; $μ=\frac{1}{N}∑_{i=1}^Nx_i$

I have noticed that this definition introduce a systematic negative bias to the results due to the subtraction of the sample mean. I calculated the auto-correlation of a white noise sample and after repeating that for thousands of realizations and averaging them in each time lag, I got that bias (attached are my code).

I was really surprise at first, but then, I calculated $E(AC_x(l))$ ($E$ for expectancy value) by hand and got: $E(AC_x(l))=-(1-\frac{l}{N})E(x_i^2)$ $^1$ (for $l>0$). This is the exact pattern I got in my simulations.

So, my question is how should I interpret that? And which definition I should use for auto-correlation in order to get the intuitive result of $\delta(l)$?

def autocorrelation(x):
    x_mean = np.mean(x)
    x_var = np.sum((x-x_mean)**2)
    corrs = []
    for i in range(len(x)):
        test_x1 = x[:(len(x)-i)]
        test_x2 = x[i:]
        corrs.append(np.sum((test_x1 - x_mean) * (test_x2 - x_mean)) / x_var)
    return np.array(corrs)
s = np.random.normal(size=(10000, 10))
corrs = np.array([autocorrelation(s[i]) for i in range(s.shape[0])])
plt.figure(figsize=(4*3, 3*3))
plt.errorbar(np.arange(1, corrs.shape[1]), np.mean(corrs[:, 1:], axis=0), st.sem(corrs[:, 1:], axis=0))
plt.xlabel('time lag')
plt.xticks(ticks=np.arange(1, corrs.shape[1]))

enter image description here

$^1$ A full proof of the bias expression:

\begin{align} E(AC_l (x))&=E(\sum_{i=1}^{N-l}(x_i-μ)(x_{i+l}-μ))\\ &=\sum_{i=1}^{N-l}E((x_i-μ)(x_{i+l}-μ))\\ &=\sum_{i=1}^{N-l}E(x_i x_{i+l}-x_i μ-x_{i+l} μ+μ^2 )\\ &=\sum_{i=1}^{N-l}E(x_i x_{i+l} )-E(x_i μ)-E(x_{i+l} μ)+E(μ^2 )\\ &=\sum_{i=1}^{N-l}E(x_i x_{i+l})\\ &\phantom{=\sum_{i=1}^{N-l}}-E\left(x_i\frac {x_1+⋯+x_N }{N}\right)\\ &\phantom{=\sum_{i=1}^{N-l}}-E\left(x_{i+l}\frac {x_1+⋯+x_N }{N}\right)\\ &\phantom{=\sum_{i=1}^{N-l}}+E\left(\left(\frac {x_1+⋯+x_N }{N}\right)^2\right)\tag{a} \end{align}

Now, for $i\ne j$: $E(x_i x_j )=E(x_i )E(x_j )=0$, because they are independent and each with 0 mean, therefore a lot of mixed elements will fall off, and we will stay with:

\begin{align} &=\sum_{i=1}^{N-l}\left(-E(\frac{x_i^2}{N})-E(\frac{x_{i+l}^2}{N})+E(\frac{x_1^2+⋯+x_N^2}{N^2})\right)\tag{b}\\ &=\sum_{i=1}^{N-l}\left(-\frac{E(x_i^2 )}{N}-\frac{E(x_{i+l}^2)}{N}+\frac{E(x_1^2)+⋯+E(x_N^2)}{N^2}\right)\tag{c} \end{align}

$x_i$ are identically distributed, therefore:

\begin{align} &=\sum_{i=1}^{N-l}\frac{-NE(x_i^2 )-NE(x_i^2 )+NE(x_i^2 )}{N^2}\\ &=\sum_{i=1}^{N-l}-\frac{E(x_i^2 )}{N}\tag{*}\\ &=-\frac{N-l}{N} E(x_i^2) \\ &=-(1-\frac{l}{N})E(x_i^2) \end{align}

And this is exactly what the simulation shows for N=10

  • $\begingroup$ if your noise is white, then $\mu\equiv 0$. Otherwise, the noise wasn't white. Attention, you'd need an infinitely large $N$ to really measure that, though the error gets smaller proportionally to $N$. $\endgroup$ – Marcus Müller Jan 1 '20 at 11:30
  • $\begingroup$ @MarcusMüller Thanks a lot for your response. 1. I was asking on the interpretation of the auto-correlation of a finite sample of some random process (white noise, for example). At first, I was expecting to get a delta function, but actually I got a non-zero bias (that can be mathematically expressed, as I showed). 2. This negative bias does not get smaller as N get larger, it's actually increases (in absolute term) and converges to -1. I would be very glad to hear your interpretation for finite sample auto-correlation, or another valid way to calculate it. $\endgroup$ – Tomer Milo Jan 1 '20 at 13:11
  • $\begingroup$ :) I'd like to, but really, the expectation of the autocorrelation of a white series is 0, per definition of "white" for any non-zero lag $l$ (there, it's its Variance). Could you maybe add the calculation that led to your expectation value to your question? $\endgroup$ – Marcus Müller Jan 1 '20 at 13:42
  • $\begingroup$ @MarcusMüller, I attached the mathematical "proof" above in the main text cause it was too long for comment. Thanks a lot! $\endgroup$ – Tomer Milo Jan 1 '20 at 14:55
  • $\begingroup$ beautified your equations a bit :) Let me have a look at it, might take a few minutes, though. $\endgroup$ – Marcus Müller Jan 1 '20 at 15:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.