Does sum of random variables follows gaussian distribution although each have different mean and variances?

Question

I understand Central limit theorem, but cannot understand the result when it applies to diverse situation.

As far as I understand, Central limit theorem states follows:

• No matter what random variable has its distribution, if we pick n samples and mean (or just sum) them and do this many time, those values will follow normal distribution (if n is big enough).

And there is a good example for this: a Brownian motion. If we measure the distance that particles moved suspending in a fluid, distance will follow normal distribution because sum of random hit(movement) of fluid particle will follow normal distribution according to Central limit theorem.

However, for central limit theorem applies, random variables which are merged should follow all the same means and variances. In upper example, all random particle movement will follow same mean and variances, because they have same temperature.

But as far as I understand, there are many cases where noise follows normal distribution but there components are not necessarily follow same mean.

(S_n = (X_1 + X_2 + ... X_n) / n and S_n follows normal distribution but X_i s have all DIFFERENT means)

For example, electrical noise will be composed of many other things which are not necessarily follow equal to each mean. But why electrical noise can be approximated as gaussian noise? Not only electrical noise, why many other things are treated as normal distribution(S_n follows normal distribution) although their components don't have equal mean?(X_i s have all DIFFERENT mean and variances)

Thank you for reading!

Answer 1

You can add random variables with different means and variances. For example, look at two continuous uncorrelated normal-distributed random variables $x_1$ and $x_2$ with means and variances $\mu_1, \sigma_1^2$ and $\mu_2, \sigma_2^3$

Then $x_3 = x_1 + x_2 $ is also a normal-distributed random variable with $\mu_3 = \mu_1 + \mu_2$ and $ \sigma_3^2 = \sigma_1^2 + \sigma_2^2 $. For uncorrelated random variables the means and variances just sum.

The distribution of a sum of two continuous random variables is the convolution of the individual distributions. That's actually the core of the central limit theorem. If you convolve a Gaussian with a Gaussian you get another Gaussian (see for example https://jeremy9959.net/Math-5800-Spring-2020/notebooks/convolution_of_gaussians.html) . If you keep convolving reasonably looking functions with each other eventually it will approach a Gaussian.

A simple example is summing a few uniformly distributed random variables. The first one has a rectangular distribution. The second one is triangular. The third one already shows a nice round belly in the middle and once you are up to summing 10 or 12 it looks very much Gaussian. That's in fact one method how gaussian random number generators are implemented.

For the Central Limit Theorem, there is no requirement for random variables to have the same mean or variance.

Answer 2

I think we need to be a fair bit more specific:

No matter what random variable has its distribution,

… If that distribution has a finite variance and mean (counterexample: Cauchy-distributed variables, e.g. $\operatorname*{Im}(z)/\operatorname*{Re}(z)$ of complex normal $z$),

if we pick n samples and mean (or just sum) them and do this many time, those values will follow normal distribution (if n is big enough).

It's called central limit theorem for a reason: the distribution of the of the sum of normalized, centralized iid RVs, divided by the square root of the number of summands, converges against the standard normal distribution:

$$ \lim_{n\to\infty}P\left(\frac{1}{\sqrt{n}}\sum_{i=1}^n (X_i-E(X_i))/\sigma_i \le x\right)=\Phi(x)$$

(and if you don't want to normalize, just multiply both sides of that inequality with a factor of the $\sqrt{\sum\sigma_i^2}$; you can work with noncentrals simply by adding to the value: Things are nice and linear)

That doesn't mean it will be exactly a normal distribution for any finite $n$. I mean, you wouldn't expect the sum of $n=3$ random variables to be normally distributed, right. There's some distributions where the convergence speed is high, there's others where it's low.

$$S_n = \frac{X_1 + X_2 + \ldots + X_n}n$$ and $S_n$ follows normal distribution but $X_i$ have all DIFFERENT means

Absolutely no problem here! When summing up two independent random variables (RVs) with probability density functions (pdf) $f_a$ and $f_b$, respectively, then the sum has a density that's the convolution of each.

Now, a normal distribution with variance $\sigma_i^2$ and mean $\mu_i$ has pdf

$$f_i(x) = \frac{1}{\sigma_i \sqrt{2\pi} } e^{-\frac{1}{2}\left(\frac{x-\mu_i}{\sigma_i}\right)^2},$$

and if you convolve two of these, you'll see that the convolution is again a normal variable, with $\sigma_{\Sigma}^2=\sigma_1^2+\sigma_2^2$ and $\mu_\Sigma=\mu_1+\mu_2$; you need no CLT for that at all, that's just stubborn evaluation of the convolution (or going to Fourier domain, doing a multiplcation there, and then back, which is generally considered less work).