Determining the uncertainty of an autocorrelation

Question

My problem should probably be built up from the beginning, so lets start there. I performed a certain experiment 25 times. Every time, the experiment consists of 5000 measurements, and each measurement returns either 1 ('yes') or 0 ('no'). I know that each time I measure 1, the probability that this measurement is correct is $F_1$, while if I measure 0 I know that it's correct with probability $F_0$.

Now, subsequently I want to compute the autocorrelation function of these measurements I use the following formula

with a slightly different normalization, but in principle that should not matter, because what I want to do is fit this autocorrelation to an exponential decay and find the decay time. To do so, I thought I had two options: Either I fit the 25 datasets separately, and then find the mean of the decay time $t_1$ and maybe say something about its variance, or I can find the mean autocorrelation of the 25 datasets, and fit that one. I've decided that that is probably the best option, as the individual datasets can be quite noisy with strange fluctuations, while the mean looks much cleaner.

But now I'm stuck wondering what I should be doing to find the uncertainty in the fit. Should I have introduced some sort of uncertainty due to the imperfect measurements ($F_0$ and $F_1$), or should I have introduced some sort of standard error of the mean for the 'mean autocorrelation' data? That last one seems plausible, as of course there is also a standard deviation in the 25 datasets, but then I get a little confused as to how I should do that. The formula for the standard error of the mean seems to be $\frac{\sigma}{\sqrt{n}}$, but this is a bit unfair as in my 5000 measurements, the autocorrelation drops to 0 after around 50 measurements. For the fit I therefore also only use the first ~100 points of the autocorrelation, as there's no real need to fit the next 4000 points that are all pretty much equal to 0. So should I instead only calculate the mean and standard error of the first 100 points? Would that be a fair way of finding the uncertainty? The subsequent fitting method will already give error bars for the $t_1$, but this of course heavily depends on the uncertainty in the data.

A final sort of sidetrail, what kind of quantitative method of finding how good I fit should I use for exponential decay? Reduced Chi square, Adjusted R squared, or something else entirely? I suppose this is not intended for dsp though, more for a statistics forum, so feel free to ignore it.

Answer 1

As far as I can see the main question is: How to account for the uncertainty in the measurements of $X_t$ (i.e. $F$).

One approach to do this would be to run some sort of Monte Carlo simulation on the data. I.e. for each experiment you can resample a large number of times (say 1000 or 10000) where each point has $1-F$ probability of being flipped. By calculating the auto-correlation of each resample you get a distribution of the correlation for each $k$. Then take the 95% (or another value) confidence interval to get an uncertainty in $R(k)$.

It may also be possible to do this analytically but I'm not sure for binary data (my instinct says not or at least not simply).

This uncertainty in $R$ should be accounted for in your fitting of the data when calculating the decay time. This can definitely be done analytically and may be implemented in whatever you are using to do the fit (I think matlab does). Alternatively a similar Monte Carlo simulation could be done but is probably more effort.

If you think all the data follows the same trend (not unreasonable if it is a repeated experiment) you could calculate a mean for each auto-correlation distance as you suggest. This is effectively taking a manual Monte Carlo with 25 resamples. Then you can calculate the standard error as you suggest (note: $n=25$, not $5000$). I would shy away from this approach as it ignores the uncertainty in you initial measurement which you presumably have a good reason for giving.