# Why does signal averaging reduces noise levels by more than $\sqrt{n}$?

I have an electrophysiology signal, which is time-based. It basically measures neural activity in the form of potential differences over time.

The noise in this signal is assumed to be a random background noise caused by the device electronics and is assumed to have a normal (i.e., Gaussian) distribution (based on previous literature) and it is supposed to be stationary. Below I have added some plots that show a raw sweep (2.3 ms of recording per sweep, sample rate= 56k, 127 points were stored), the histogram of the values, and the FFT.

In answer to the answer below: I am fairly confident we can assume normally distributed data, since it's a bell shape. However, The FFT is not flat as answerer suggests it should be when normally distributed. However, we only have a few ms of data, which may complicate this type of analysis.

Assuming a normal distribution, theoretically, doubling the number of sweeps and averaging them should reduce the noise by a factor of the square root of $2$ (about $~1.41$). This kind of signal averaging to reduce random background noise is called ensemble averaging.

Now, I have a recording (without a signal, just background) and I have determined the noise level at $1$, $2$, $4$, $8$ and $16$ averages.
Edit: This was done by re-recording the background noise and determining the average with MATLAB, in the format B = mean(A1,A2), with Ai being a vector with a time-based signal.

Noise level was defined as the standard deviation of the sweep. When I determine the factor of improvement between these averages, i.e., 2 vs 1, 4 vs 2, 8 vs 4, and 16 vs 8, I find the factors $1.4$, $2.5$, $2.5$ and $2.3$, respectively. These improvement factors were simply calculated by determining the ratio between SD$n$ / SD$2n$, with $n$ being the number of averaged sweeps.

The factors I find are averages across 31 electrodes measured in a total of N = 13 people. In other words, it's consistent across trials and 13 different devices. I didn't do stats, but I bet the latter three improvements are significantly $> 1.41$.

Why is the noise reduction higher than the theoretical 1.41 when doubling my sweeps from 2 to 4, from 4 to 8, and 8 to 16?

Raw sweep; 2.3 ms, 56 kHz sample rate, 127 data points

Histogram of amplitude distribution

FFT of the signal

• Interesting. Could you be more specific on how you perform the said averages on one recording? – Laurent Duval Mar 30 '18 at 12:42
• And when you perform 4 or 8 fold averaging, is that mean of mean (of mean)? So you have at least 16 background signals? If you are ok to share your data and code, you might get other insights – Laurent Duval Mar 30 '18 at 13:40
• I don't quite get "N = 13 people, across 3 electrodes (n = 31 electrodes)" yet, if you could detail a little more – Laurent Duval Mar 30 '18 at 13:51
• @LaurentDuval - I've added some graphs too. – AliceD Mar 30 '18 at 14:19
• Excellent, more over the week-end – Laurent Duval Mar 30 '18 at 14:23

Well, I would say the assumption that your noise is Gaussian is ill fitting. If the noise is due to machine interference, it probably has some tonal characteristics. Tones of the same frequency can reinforce or cancel each other out when added.

To get a better idea of what might be going on, you should:

1) Make a histogram of the noise

2) Take an FFT of the noise

The histogram should look like a bell curve and the FFT results should be flat. If they aren't, your noise is not Gaussian.

Hope this helps.

Ced

• I have added sample data in the question, including a histogram and FFT. I'm unsure whatto make of the FFT at this stage. Problem is the sweeps are just 2.3 ms long. – AliceD Mar 30 '18 at 14:20
• @AliceD, The first chart of your raw data show you clearly don't have Gaussian white noise, which is the assumption for the sqrt(2) rule. The FFT confirms it. Your distribution is Gaussian shaped, except it is not zero centered. I am interested in what Laurent Duval has to say. Thanks for posting the charts, this is an interesting question. – Cedron Dawg Mar 30 '18 at 14:31
• Thanks so much - it is indeed not zero-centered; other recordings show predominantly negative values too. That has to do with the baseline - we did low-pass filter our data, but no high-pass filtering whatsoever. – AliceD Mar 30 '18 at 14:35
• But why would 0-centering be necessary? Many distributions with a mean deviating from 0 are normally distributed? – AliceD Mar 30 '18 at 14:53
• With baselines coming into play, and potential histogram asymmetry, many interesting stuff can show – Laurent Duval Mar 30 '18 at 14:56

In general, for any distribution of $x_i$ that has a variance

$$\text{The sum} \sum_{i=0}^{N-1}x_i \;\text{has variance} \;\sigma^2_{total}=\sum_{i=0}^{N-1} \sigma_i^2$$ if the $x_i$ are independent. $$\text{The average}\frac{1}{N} \sum_{i=0}^{N-1}x_i \;\text{has variance} \;\sigma^2_{ave}=\frac{1}{N}\sum_{i=0}^{N-1} \sigma_i^2$$ which in terms of the question. $$\sigma_{ave} = \frac{\sigma_i}{\sqrt{N}}$$ if the $x_i$ have the same variance, or in other terms is iid (independent identically distributed)

Gaussian or not, this is true, and it doesn't depend on the Central Limit Theorem. It just has to be iid. Gaussianity is irrelevant.

When $E\{x_i\} \ne 0$ you need to be careful when calculating the standard deviation. $$E\{x_i^2\}= \sigma^2 + E\{x_i\}^2$$ For sums of independent random variables, variances add if they have a variance. As an example a Cauchy distribution doesn't have a variance.

If your measurement is of the form $r_i=s_i+n_i$ where $s_i$ is a deterministic signal and $n_i$ is iid noise. The calculation of SNR gain is straightforward if you are forming ensembles with exactly the same $s_i$ and are exactly aligned when averaging. If $s_i$ is random (or partially random) the calculation is more complicated.

• What you said is true but without the normality assumption ( or the large n assumption ), the averages won't be normal. But maybe he doesn't need that to be true. That's where my lack of details of what he's doing comes into play. Thanks. – mark leeds Mar 31 '18 at 0:16
• Also, one more thing: Without the assumption about $\sigma_{x}$ being known, in your descrpption, what would he use for an estimate of $\sigma_{x}$. Without large $n$ or known $\sigma_{x}$, a constructed estimate of it is tenuous at best. – mark leeds Mar 31 '18 at 0:23
• The question was about the SMR Gain as sqrt(n). The SNR is a ratio of expected power values. Estimates of SNR are governed by the the law of large numbers, weak or strong. As a worst case there is the Chebychev bound. If the OP posted his code, the problem could probably be tracked down. – user28715 Mar 31 '18 at 0:51
• OK.. Gotcha. I'm not familar with SNR gain so I just read it as he want to reduce some kind variation by averaging (which might be incorrect ?). It's just not clear to me. based on your explanation, where he would obtain $\sigma_{i}$ which is my $\sigma_{x}$. In my case, I assumed its known because, if you have to estimate it, you've got to make more assumptions that you probably want to. At the same time, you've gotta to get it from somewhere so maybe estimation using the sample variance is the only way. Thanks. – mark leeds Mar 31 '18 at 2:16
• I guess if you have a statistics background, if you see a parameter, you got to estimate it. Gain is more about making what you want bigger relative to what you don’t want. If the noise is IID and has a variance, coherent averaging reduces that noise power by sqrt(N). Down stream, knowing the value of that parameter will be handy for something like setting a threshold but in real life, a criteria like operator work load will often set thresholds – user28715 Mar 31 '18 at 3:10

Hi: I don't follow the whole thing ( not even close. terminology doesn't work for me ) but you may be interpreting the square root of n rule incorrectly. The square root of n rule really means the following statistically.

Suppose I had a normally distributed random variable, $x$ whose sd was known to be $\sigma_{x}$. ( notice I said $\sigma$ is known and not estimated ) and known mean whatever ( take it as zero but it doesn't matter ). So
the $x_{i}$s come from the normal distribution with mean zero and sd $\sigma_{x}$.

Experiment: Generate $n~ x_{i}$'s and calculate the average: $\bar{x}_{1}$. Again, generate $n~ x_{i}$'s and calculate the average $\bar{x}_{2}$. Do this over and over say $N$ times so you get $N$ random variables, $\bar{x}_{i}, \ldots \bar{x}_{N}$ each of which is an average of $n$ observations.

Then, the correct "square root of n" statement is that the $\bar{x}_{i}$ ( there are N of them but now one can think of them as coming from a population ) have a normal distribution with the same mean as the original mean ( so zero ) and standard deviation $\frac{\sigma_{x}}{\sqrt{n}}$

The confusion may stem from the fact that are variations of this statement that use the CLT to arrive at the same conclusion but the CLT needs large n to ensure convergence and things get much more fuzzy once you need CLT so that's where your mis-understanding ( if there is one) may come from.

If $\sigma_{x}$ is known and the underlying distribution is normal, then the CLT is not needed as an assumption and this statement is fact and independent of the value of $n$ ( CLT versions need large n and don't assume $\sigma_{x}$ known so you need to assume that the estimated value of $\sigma_{x}$ has converged to the true one ). You can try the experiment out in matlab or R and see for yourself. If I had time, I'd show it in R but I don't.

Like I said, I don't follow what you're doing but this may be where the weirdness is coming from. I hope this helps.