Gaussian White Noise - Relation Between Distribution and Correlation

Im a beginner in signal processing so my question may be obvious.

A white noise has the property to have its autocorrelation function that is equal to

$$\mathbb{E}[f(t+\tau)f(t)]=\sigma^2 \delta(\tau).$$

We say that random signals can be Gaussian white noise, but for me if the signal has a Gaussian distribution it is necesseraly a white noise because as soon as we know the distribution then $\mathbb{E}[f(t+\tau)f(t)]$ is automatically determined.

Am I wrong ?

When we say the stochastic process $X_{t_1}X_{t_2}\cdots X_{t_i}$ is a Gaussian Process with mean $\mu$ and variance $\sigma^2$ it only means $$X_{t_i}\sim\mathcal{N}(\mu,\sigma^2)$$ So still we know nothing about the relation of the $X_{t_i}$'s with each other. When we add that the process is also white, it means the $X_{t_i}$'s are uncorrelated. That is

$$\mathsf{E}[(X_{t_i}-\mathsf{E}X_{t_i})\mathsf{E}(X_{t_j}-\mathsf{E}X_{t_j})^*]=\begin{cases}\sigma^2,&i=j\\0,&i\neq j\end{cases}$$

Off topic, but notice that the $\tau$ notation is an implicit assumption of the process being stationary.

The process can be Gaussian but colored (rather than white). In such case, the random variables are Gaussian but correlated. For instance, the autocorrelation function in case of stationary Gaussian process you mentioned can be something like this: $$R_{XX}(\tau)=\sigma^2e^{-|\tau|}$$

• White noise is always assumed to have zero mean, even in the continuous-time case. Absent this property, the power spectral density has an impulse at the origin, and thus the PSD does not have the defining characteristic of white noise: constant value for all frequencies. – Dilip Sarwate Nov 24 '16 at 21:56
• Thank you ! The thing that I don't really get in fact is that i a process at time "n" follows a given law of probability I don't see how it could depend of what happened before ? Is it smth like this : I have 10 card with number s from 1 to 10 and 1 chance over 10 to have a given one. Then If I do the experiment 10 times the probability will always be the same. But If I decide to say "if I have the number "x" at a previous experiment, then I put it at the back of my set (in position 10), then there will have a correlation between two experiment ? – StarBucK Nov 24 '16 at 21:57
• Yes @DilipSarwate, thank you for adding that. user3183950: I couldn't fully catch your example, but assume a stochastic process $X_{i+1}=aX_{i}+Y$, where $Y$ is another Gaussian random variable, independent of $X_{i}$ and $X_{i+1}$. You can see the process $\{X_{n}\}$ is Gaussian but the samples are correlated. – msm Nov 24 '16 at 22:51

We say that random signals can be Gaussian white noise, but for me if the signal has a Gaussian distribution it is necesseraly a white noise because as soon as we know the distribution then $$\mathbb{E}[f(t+\tau)f(t)]$$ is automatically determined.

Am I wrong ?

Yes, you are wrong in various ways.

First, by Gaussian noise is meant a random process (not a random function as the OP calls it) -- that is, an infinite collection of random variables, one random variable for each time instant $$t$$ -- such that if we pick any $$n \geq 1$$ random variables $$X_{t_1}, X_{t_2}, \ldots, X_{t_n}$$ from the random process, then these random variables have a $$n$$-dimensional joint Gaussian distribution. Now, a $$n$$-dimensional Gaussian distribution is entirely determined by the $$n$$ means of the random variables and the $$n\times n$$ covariance matrix of the random variables, but note very carefully that nothing at all has been asserted about the means or the covariance matrix of the $$n$$ Gaussian random variables. Specifically, nowhere is it written that all the means must be the same or that $$X_{t_i}$$ and $$X_{t_j}$$ have the same variance or that $$\operatorname{cov}(X_{t_i},X_{t_j}) = \operatorname{cov}(X_{t_i+1},X_{t_j+1})$$ etc. All that is meant by Gaussian noise is that the $$n$$ random variables have a jointly Gaussian distribution (which implies, among other things, that $$X(t)$$ and $$X(s)$$ individually are Gaussian random variables as well).

Second, white noise is a mythical process that is unobservable in all its glory in nature (probably just as well since it is infinitely powerful and would lead to an immediate solution to the energy crisis). We poor mortals can only observe white noise through some kind of device that necessarily limits what we can observe -- kind of like watching a solar eclipse through special glasses-- and thus what we observe is a pale imitation of the real thing. Well, it has been observed that if an observation device is modeled as a linear filter with transfer function $$H(f)$$, then (with an open circuit at the filter input), the filter output is a random process with power spectral density $$\sigma^2|H(f)|^2$$. This is consistent with an assumption that the input to the filter is a white noise process with autocorrelation function $$\sigma^2\delta(t)$$ where $$\delta(t)$$ is a Dirac delta or impulse and power spectral density $$S(f) = \sigma^2, -\infty < f < \infty$$ if we simply plug in $$\sigma^2$$ for the input power spectral density in the power spectral density equation $$S_{\text{output}}(f) = |H(f)|^2 S_{\text{input}}(f).$$ Never mind that many mathematicians will cringe at the cavalier treatment where we are ignoring that the above formula implicitly assumes that the input process is a finite power process (which white noise is definitely not); we are engineers and we don't care. We are used to treating impulses like ordinary functions.

Finally, turning to discrete-time random processes, remember that one cannot sample the mythical beast called white noise -- it does not exist in nature -- and the sampler is necessarily a device that observes the random process for a very short but nonzero time $$\varepsilon$$, and thus the sample $$X[n]$$ is actually something proportional to $$\int_{nT-\varepsilon/2}^{nT+\varepsilon/2}X_t \mathrm dt$$ which has variance $$\sigma^2\varepsilon$$ if $$\{X_t\}$$ is a white noise process. So,

A discrete-time white noise process is a collection of zero-mean independent identically distributed random variables $$X[n]$$.
A discrete-time white Gaussian noise process is a collection of zero-mean independent identically distributed Gaussian random variables $$X[n]$$.

Yes, many DSP texts (as well as Wikipedia's definition of a discrete-time white noise process) and many people with much higher reputation than me on dsp.SE say that uncorrelatedness suffices for defining a white noise process, and in the case of white Gaussian noise it does because Gaussianity brings in the jointly Gaussian property: a discrete-time Gaussian random process is defined as a sequence of random variables $$\{X[n]\colon n \in \mathbb Z\}$$ such that any set of $$M\geq 1$$ random variables $$X[n_1], X[n_2], \ldots, X[n_M]$$ enjoys a jointly Gaussian distribution, and so for white Gaussian noise, uncorrelatedness implies independence. However, for arbitrary white noises, it is best to insist on independence and not on just zero correlation. For the edification of all these important people who insist that uncorrelatedness is adequate, I present a discrete-time process in which every random variable is a Gaussian random variable, any two random variables are uncorrelated but are not necessarily independent, and not all sets of variables in the process enjoy a jointly Gaussian distribution. In short, the process is not a white Gaussian noise process as per the standard definition. And why should all this matter in the least? Well, in typical applications we apply various mathematical operations on processes, and if $$X[0]$$ and $$X[1]$$ are uncorrelated Gaussian random variables and we cannot rely on $$X[0]+X[1]$$ also being a Gaussian random variable, things have come to pretty pass, and it is not a world I want to live in.

Example: Let $$X$$ be a $$N(0,1)$$ random variable and $$B$$ a discrete random variable that takes on values $$+1$$ and $$-1$$ with equal probability $$\frac 12$$ and independent of $$X$$. Set $$Y = BX$$ and note that $$E[Y]=E[BX]=E[B]E[X]=0$$. Furthermore, $$E[XY] = E[X^2B] = E[X^2]E[B] = 0$$, and so $$X$$ and $$Y$$ are uncorrelated random variables. But what is the distribution of $$Y$$? Well, \begin{align} P(Y \leq a) &= P(Y\leq a \mid B=+1)P(B=+1) + P(Y\leq a \mid B=-1)P(B=-1)\\ &= \frac 12 P(BX\leq a \mid B=+1) + \frac 12 P(BX\leq a \mid B=-1)\\ &= \frac 12 P(X\leq a) + \frac 12 P(X\geq -a)\\ &= \frac 12 \Phi(a) + \frac 12 \Phi(a)\\ &= \Phi(a), \end{align} that is, $$Y$$ is also a $$N(0,1)$$ random variable!! But $$X$$ and $$Y$$ are not jointly Gaussian random variables. Note that conditioned on the value of $$X$$ being $$\alpha$$, $$Y$$ is a discrete random variable that takes on values $$\pm\alpha$$ with equal probability: with joint Gaussianity, $$Y$$ would have been a Gaussian random variable.
With this as background, let $$\{X[2n]\colon n \in \mathbb Z\}$$ be a set of independent identically distributed zero-mean Gaussian random variables, that is, a standard white Gaussian noise process on the even integers. Let $$\{B[n]\colon n \in \mathbb Z\}$$ be an independent process where the $$B[n]$$'s are independent discrete random variables that take on values $$+1$$ and $$-1$$ with equal probability $$\frac 12$$. Set $$X[2n+1] = X[2n]B[n]$$ and note that each pair $$(X[2n],X[2n+1])$$ is a pair of uncorrelated zero-mean Gaussian random variables that are not jointly Gaussian. Now let's look at the random process $$\{X[m]\colon m \in \mathbb Z\}$$ in which all the random variables are zero-mean Gaussian with the same variance. Any pair of random variables is uncorrelated: $$X[2n]$$ and $$X[2n+1]$$ by construction and all the more distant pairs because of independence. But, not all pairs of random variables have a jointly Gaussian distribution and so this is not a white Gaussian noise process in the usual sense of the term; ymmv.

• Reading through your amazing answers has got me to this post. I have a doubt: Why did you mention that for arbitrary White noise we should insist on independence and not just 0 correlation? Wouldn't 0 correlation mean that the auto-correlation is a delta function and the Noise PSD is constant, hence noise is white? I can think of one reason that for non Gaussian noise, whiteness will not imply independence. So, the non Gaussian white noise will still be difficult to work with. Is that the idea? – DSP Rookie May 27 at 20:03
• @DSPRookie The Wikipedia page on white noise says in the part about white noise vectors that "Often the weaker condition "statistically uncorrelated" is used in the definition of white noise, instead of "statistically independent". However some of the commonly expected properties of white noise (such as flat power spectrum) may not hold for this weaker version." and that's what I use. The Truth-in-DSP Act compels me to tell you that in the next section, discrete-time white noise is defined as a sequence of uncorrelated .... – Dilip Sarwate May 27 at 21:10
• @DSPRookie No, the process that I have constructed is not a white noise process according to what I claim is the correct definition, but it is a white noise process according to the Wikipedia definition which is, in my opinion. dead wrong. Note that Wikipedia doesn't even need the $X[n]$ to have the same distribution! Your last sentence is missing the whole point of my example: all the $X[m]$ are individually zero-mean Gaussian and are pairwise uncorrelated but $X[2n]$ and $X[2n+1]$ are not independent: their joint pdf is not the product of the individual pdfs – Dilip Sarwate May 28 at 14:38
• @DSPRookie (continued) Essentially, the question gets down to what one thinks white noise means. To me and many others of similar mind, white noise is the most unpredictable of all possible noise processes, and no amount of previous observation should provide any clue as to what the next value is going to be. Independence of the $X[m]$ guarantees this; uncorrelatedness does not. In my example, if we observe the value $\alpha$ of $X[2n]$, we know for sure that $X[2n+1]$ equals $\pm\alpha$. The process I have constructed is not white noise as far as I am concerned and Wikipedia is wrong. – Dilip Sarwate May 28 at 14:49
• Thanks a lot for the clarification. I get it now. The main point in your example is even though $X$ and $Y$ are individually Gaussian and uncorrelated, they are not independent. When we say uncorrelatedness implies independence, that is for Jointly Gaussian RV pair to start with. But here $X$ and $Y$ are not jointly Gaussian and hence uncorrelatedness does not imply independence. – DSP Rookie May 28 at 15:07

Yes, that's wrong. Here is a counterexample. We can form a periodic signal (Fig. 1) with a Gaussian distribution:

$$f(t) = \Phi^{-1}\big(\operatorname{frac}(t)\big),\tag{1}$$

where $$\Phi^{-1}$$ is the inverse cumulative distribution function of the standard $$(\mu = 0,\,$$ $$\sigma^2 = 1)$$ normal (Gaussian) distribution, and $$\operatorname{frac}(t),$$ with $$0 \le \operatorname{frac}(t) < 1,$$ gives the fractional part of the real number $$t.$$ The function $$\Phi^{-1}$$ is also called the probit function. It can be used to convert numbers coming from the standard uniform distribution into the standard normal distribution (see Inverse transform sampling).

Figure 1. The periodic signal $$f(t).$$ Each period equals the probit function. At integer $$t$$ the function approaches $$\pm\infty.$$

Because the signal is periodic with period 1, then:

$$f(t + 1) = f(t)\quad\text{for all }t\in\mathbb{R}.\tag{2}$$

It follows that:

$$\mathbb{E}[f(t+\tau+1)f(t)]=\mathbb{E}[f(t+\tau)f(t)].\tag{3}$$

If the signal would be white, then, plugging in from the question the equation $$\mathbb{E}[f(t+\tau)f(t)]=\sigma^2 \delta(\tau)$$ for the autocorrelation function of white noise, we would get:

$$\sigma^2\delta(\tau + 1)=\sigma^2\delta(\tau)\tag{4}$$

and with non-zero $$\sigma^2,$$ for $$\tau = 0:$$

$$\delta(1) = \delta(0),\tag{5}$$

which is not true according to the definition of the Dirac delta function. The conclusion is that the signal $$f(t)$$ cannot be white.

Periodic signals are composed of harmonic frequencies only.

The plot was generated in Octave by:

x = ([-200:299]+0.5)/100;
y = (sqrt(2)*erfinv(2*(x-floor(x))-1));
plot(x, y);


Gaussian is for the distribution, but white for the correlation. When we say the sample $$z_1,\,z_2\,\cdots,\,z_n$$ are white Gaussian random variables, it means that each sample follows a Gaussian distribution with certain mean and variance, but the samples are uncorrelated and thus independent. The samples could be Gaussian be not white (it's called colored), in which case the samples are correlated.