2
$\begingroup$

After reading two articles on signal processing stack exchange:
On coloured Gaussian noise
How the white and colored noise differ in time domain

I do understand that variance do not change over frequency (or time) for colored Gaussian noise.

Let's say I have a random process $y_n$, which is defined as: $$ y(n) = x(n-1) - x(n), \quad\text{where }x_n\text{~} N(0, \sigma_n^2) $$ For the clarification, $x_n$ is white Gaussian, and hence $E[x_n\cdot x_m]=0 \text{ for } n\neq m$.
Here, we can see that PSD of $y_n$ has high-pass form since $R_{yy}(\tau)=2R_{xx}(\tau)-R_{xx}(\tau-1)-R_{xx}(\tau+1).$ we can also easily see that its variance is $2\sigma_n^2$.

Next, we also introduce another random process which has a 2x higher variance than $x_n$: $w_n\text{~}N(0,2\sigma_n^2)$

Now, I would like to compare the random process $y_n$ to $w_n$. First, we know that their variance are identical to $2\sigma_n^2$. Second, although $y_n$ is colored Gaussian and $w_n$ is AWGN, their PDF must look the same, because they are still Gaussian process.

What I do not understand clearly is the meaning of PSD of $y_n$. As shown below, plot with a red color is a PSD of $y_n$. It shows high-pass shape, start having higher power magnitude after passing $0.5 f_s$. But what does it mean for a Gaussian noise having higher power magnitude at high frequency and having lower power magnitude at low frequency?

It cannot be that $y_n$ has higher "variance" at high frequency and lower "variance" at low frequency because I mentioned earlier that variance should not vary over frequency or time.

The next thing I thought about is the difference between certain sequences of $y_n$ over time. For instance, if $y_n$ has a high frequency pattern (i.e., transition density of 1) for $n=1\cdots 100$, then it may temporarily have a higher variance for this particular high frequency pattern. But soon I realized that this argument does not make sense neither since Fourier transform does not care about behavior of $y_n$ for some local pattern.

If anyone can give me any good insights about the meaning of PSD for the example above, I would be greatly appreciated!

Please let me know if anything is not clearly in the above. English is not my first language so there is a very high chance I did not explain things very clearly.

PSD comparison between <span class=$Y(f)$ vs $W(f)$" />

$\endgroup$
1
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$
    – Peter K.
    Mar 29 at 13:35
1
$\begingroup$

Think in terms of noise realizations. When your head is full to the brim of covariances and multivariate Gaussian variables, generate a Gaussian white noise realization of length N, and create, using this sequence, a sequence $y(n) = x(n) - x(n-1)$. Let us call the latter a blue Gaussian noise realization. I plot histograms of these sequencies which show how many values fall into each bin, those bins uniformly partitioning a range of $y, x$ values.

histograms

In compliance with your convention, a blue colored graph represents the white noise data, a red colored, the blue noise data. This histograms visually agree with your observation that

$w_n \sim N(0,2σ_n^2)$, 2x higher variance than $x_n$

Each histogram plotted here is an approximate representation of the distribution of noise signal data, id est, the Probability Density Function of noise signal. The realization sequence's length N is 220 samples. The longer sequence length gets and the greater bin size gets, the closer these numerical PDF graphs resemble theoretical curves.

Go on to Power Spectral Density. Here are discrete Fourier transforms (actually, FFTs) of our white-noise and blue-noise realizations, squared and plotted. Color scheme according to your convention (white noise in blue, blue noise in red):

PSD

A white-noise PSD (blue graph) is near-constant, a blue-noise PSD graph (red graph) is two symmetrical parabolic segments sewn at 1/4 and 3/4 of the entire range (the shapes are revealing $\omega$ squared function and FFT mirroring). The high-frequency end is in the middle of horizontal axis.

No relationship between variance and PSD is revealed in our calculations and pictures. It appears that these concepts are things that can only be related indirectly, using formulas that have yet to be derived at this stage of learning.

Variance 'measures how far a set of numbers is spread out from their average value' (Wikipedia). Variance is defined in the time/space domain, and this definition never mentions frequencies or wavenumbers or Fourier transform. Although variance can be measured for any "set of numbers", in many signal processing scenarios said set of numbers is almost synonymous with sampled noise values.

A PSD thing, on the contrary, belongs in the frequency domain. The article in Wikipedia states from the beginning

According to Fourier analysis, any physical signal can be decomposed into a number of discrete frequencies

and continues

PSD of the signal describes the power present in the signal as a function of frequency. PSD of noises is a specific case of PSD of signals in general.

So variance and PSD are sort of "orthogonal" concepts; whatever, variance cannot be a function of frequency in any meaningful way, while PSD is.

Compare also the histograms of the white-noise realization PSD (top) and the blue-noise realization PSD (bottom). The white-noise PSD shape replicates, as it should, a normal distribution of source signal values. A peak of the blue-noise PSD shape shows greater share of the high-frequency components in the blue noise signal.

bluenoise

PS There is a number of Guassian Process tutorials on github and elsewhere, mostly Jupyter notebooks. See, for example, https://peterroelants.github.io/posts/gaussian-process-kernel-fitting/.

$\endgroup$
7
  • $\begingroup$ Nice answer and it is intuitive that the variance will double since the random independent processes will add in power (variance) and equivalently go up by the square root in standard deviation (where correlated processes will add in standard deviation.) The fact that we subtract instead of add the two independent samples in the filter makes no difference to the combined random variable since they will still be independent if we change the sign! So either y[n] = x[n]-x[n-1] (which is a high-pass filter) or y[n] = x[n]+x[n-1] (which is a low pass filter) will both double the variance. $\endgroup$ Mar 30 at 12:58
  • 1
    $\begingroup$ Also, it is interesting (IMO) for stochastic process learners to examine the dependence of an PSD{y(n),k} histogram (as in the last picture, where k=1) on delay k : {y(n)}(k) = x(n) - x(n-k), and to try and explain the behavior. Very instructive educational experience. $\endgroup$
    – V.V.T
    Mar 30 at 13:30
  • 1
    $\begingroup$ Yes, very good point and observation. This is somewhat related; with windowing instead of filtering in time but also interesting for stochastic process learners (not sure if you saw this already): dsp.stackexchange.com/questions/39047/… $\endgroup$ Mar 30 at 16:24
  • $\begingroup$ Thanks a lot for sharing knowledge and detailed answers. I've also come to the same results & conclusion after discussion with folks who commented on my original post. Thanks again. $\endgroup$
    – Emmmm
    Mar 30 at 21:29
  • $\begingroup$ Regarding the examination of dependence of PSD on delay k, one can try to compute the autocovariance of y (Ryy) in terms of v (Rvv), which is originally defined as white noise with variance sigma_n^2. This way, one can easily anticipate its shape of PSD in frequency domain. $\endgroup$
    – Emmmm
    Mar 30 at 23:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.