Why does signal averaging reduces noise levels by more than $\sqrt{n}$?

Question

I have an electrophysiology signal, which is time-based. It basically measures neural activity in the form of potential differences over time.

The noise in this signal is assumed to be a random background noise caused by the device electronics and is assumed to have a normal (i.e., Gaussian) distribution (based on previous literature) and it is supposed to be stationary. Below I have added some plots that show a raw sweep (2.3 ms of recording per sweep, sample rate= 56k, 127 points were stored), the histogram of the values, and the FFT.

In answer to the answer below: I am fairly confident we can assume normally distributed data, since it's a bell shape. However, The FFT is not flat as answerer suggests it should be when normally distributed. However, we only have a few ms of data, which may complicate this type of analysis.

Assuming a normal distribution, theoretically, doubling the number of sweeps and averaging them should reduce the noise by a factor of the square root of $2$ (about $~1.41$). This kind of signal averaging to reduce random background noise is called ensemble averaging.

Now, I have a recording (without a signal, just background) and I have determined the noise level at $1$, $2$, $4$, $8$ and $16$ averages.
Edit: This was done by re-recording the background noise and determining the average with MATLAB, in the format B = mean(A1,A2), with Ai being a vector with a time-based signal.

Noise level was defined as the standard deviation of the sweep. When I determine the factor of improvement between these averages, i.e., 2 vs 1, 4 vs 2, 8 vs 4, and 16 vs 8, I find the factors $1.4$, $2.5$, $2.5$ and $2.3$, respectively. These improvement factors were simply calculated by determining the ratio between SD_$n$ / SD_$2n$, with $n$ being the number of averaged sweeps.

The factors I find are averages across 31 electrodes measured in a total of N = 13 people. In other words, it's consistent across trials and 13 different devices. I didn't do stats, but I bet the latter three improvements are significantly $> 1.41$.

Why is the noise reduction higher than the theoretical 1.41 when doubling my sweeps from 2 to 4, from 4 to 8, and 8 to 16?

^{Raw sweep; 2.3 ms, 56 kHz sample rate, 127 data points}

^{Histogram of amplitude distribution}

^{FFT of the signal}

Answer 1

Well, I would say the assumption that your noise is Gaussian is ill fitting. If the noise is due to machine interference, it probably has some tonal characteristics. Tones of the same frequency can reinforce or cancel each other out when added.

To get a better idea of what might be going on, you should:

1) Make a histogram of the noise

2) Take an FFT of the noise

The histogram should look like a bell curve and the FFT results should be flat. If they aren't, your noise is not Gaussian.

Hope this helps.

Ced

Answer 2

In general, for any distribution of $x_i$ that has a variance

$$ \text{The sum} \sum_{i=0}^{N-1}x_i \;\text{has variance} \;\sigma^2_{total}=\sum_{i=0}^{N-1} \sigma_i^2 $$ if the $x_i$ are independent. $$ \text{The average}\frac{1}{N} \sum_{i=0}^{N-1}x_i \;\text{has variance} \;\sigma^2_{ave}=\frac{1}{N}\sum_{i=0}^{N-1} \sigma_i^2 $$ which in terms of the question. $$ \sigma_{ave} = \frac{\sigma_i}{\sqrt{N}} $$ if the $x_i$ have the same variance, or in other terms is iid (independent identically distributed)

Gaussian or not, this is true, and it doesn't depend on the Central Limit Theorem. It just has to be iid. Gaussianity is irrelevant.

When $E\{x_i\} \ne 0$ you need to be careful when calculating the standard deviation. $$ E\{x_i^2\}= \sigma^2 + E\{x_i\}^2 $$ For sums of independent random variables, variances add if they have a variance. As an example a Cauchy distribution doesn't have a variance.

If your measurement is of the form $ r_i=s_i+n_i $ where $s_i$ is a deterministic signal and $n_i$ is iid noise. The calculation of SNR gain is straightforward if you are forming ensembles with exactly the same $s_i$ and are exactly aligned when averaging. If $s_i$ is random (or partially random) the calculation is more complicated.

Answer 3

Hi: I don't follow the whole thing ( not even close. terminology doesn't work for me ) but you may be interpreting the square root of n rule incorrectly. The square root of n rule really means the following statistically.

Suppose I had a normally distributed random variable, $x$ whose sd was known to be $\sigma_{x}$. ( notice I said $\sigma$ is known and not estimated ) and known mean whatever ( take it as zero but it doesn't matter ). So
the $x_{i}$s come from the normal distribution with mean zero and sd $\sigma_{x}$.

Experiment: Generate $n~ x_{i}$'s and calculate the average: $\bar{x}_{1}$. Again, generate $n~ x_{i}$'s and calculate the average $\bar{x}_{2}$. Do this over and over say $N$ times so you get $N$ random variables, $\bar{x}_{i}, \ldots \bar{x}_{N}$ each of which is an average of $n$ observations.

Then, the correct "square root of n" statement is that the $\bar{x}_{i}$ ( there are N of them but now one can think of them as coming from a population ) have a normal distribution with the same mean as the original mean ( so zero ) and standard deviation $\frac{\sigma_{x}}{\sqrt{n}}$

The confusion may stem from the fact that are variations of this statement that use the CLT to arrive at the same conclusion but the CLT needs large n to ensure convergence and things get much more fuzzy once you need CLT so that's where your mis-understanding ( if there is one) may come from.

If $\sigma_{x}$ is known and the underlying distribution is normal, then the CLT is not needed as an assumption and this statement is fact and independent of the value of $n$ ( CLT versions need large n and don't assume $\sigma_{x}$ known so you need to assume that the estimated value of $\sigma_{x}$ has converged to the true one ). You can try the experiment out in matlab or R and see for yourself. If I had time, I'd show it in R but I don't.

Like I said, I don't follow what you're doing but this may be where the weirdness is coming from. I hope this helps.