Frequency magnitude distribution of noise

Question

By the Wiener-Khinchin theorem, we have a straightforward way to calculate the power spectral density for stationary noise. If we know the bandwidth of a system, we can further calculate the variance of the noise since it turns out that $v_{noise,\ RMS} = \sigma$ (standard deviation) for zero mean noise. Therefore, at each point in time we know the probabilistic distribution of noise as shown in this video:

However, can we determine anything about the distribution of noise magnitude at each point in frequency from a noise source with known power spectral density? I believe this could be related to power spectral density estimation?

This video says that power spectrum realizations are proportional to the square of the underlying PSD, though I am not sure if that is just an artifact of using periodograms. In any case, 'variance proportional to the square' is still not a precise probabilistic distribution.

Answer 1

Let me try to elaborate a bit on your questions: First, the Wiener-Khintschin-Theorem relates the autocorrelation of a stationary process to its PSD, with the relation of the Fourier Transform, i.e. $$\mathcal{F}\{r_{nn}(\tau)\}=S_{nn}(f).$$ The autocorrelation of a signal describes the correlation between two noise samples at distance $\tau$. It is just the second moment of the (joint) distribution of both noise samples. Hence, it does not really tell you something about the actual distribution. However, mostly, the noise is assumed to be Gaussian distributed with zero mean. Then, the autocorrelation completely describes the noise process.

Furthermore, in case you have AWGN, then $r_{nn}(t)=N_0\delta(t)$, i.e. the noise samples are uncorrelated and the PSD is a constant with $S_{nn}(f)=N_0$.

Your question is: What is the distribution of the noise magnitude in the frequency domain? So, you want to know $Pr(S(f)|_{f_0})$, where $S(f)$ is a realization of the noise process.

To make this more accessible (at least to me), let me switch to a linear algebra setting (i.e. the finite discrete-time case). The results can be straight-forwardly taken over to the continuos-time case. Let $\vec{n}$ be a realization of the noise process with autocorrelation $R_{nn}=E[\vec{n}\vec{n}^H]$. Let $\mathbf{F}$ be the unitary DFT matrix. So, $\vec{N}=\mathbf{F}\vec{n}$ is one noise realization in the frequency domain. Assuming that $\vec{n}$ consists of Gaussian distributed elements with zero mean, $\vec{N}$ also consists of Gaussian elements (A linear combination of two gaussian random variables remains to be Gaussian). What is the autocorrelation of the frequency-domain noise? It's given by

$$\mathbf{R}_{NN}=\mathbf{F}R_{nn}\mathbf{F}^H.$$

On the diagonal of $R_{NN}$ there is the variance of the Gaussians in the frequency domain.

So, the distribution of noise in the frequency domain remains to be a Gaussian, with variance given by diagonal on the above expression. I believe the diagonal is actually the PSD that follows from the Wiener-Khintschin Theorem. So, the final answer is:

The noise in the frequency domain is Gaussian distributed, with zero mean and variance given by the PSD of the noise. (Assuming zero-mean Gaussian time-domain noise)

Answer 2

In the video that OP linked, as well as here (pg. 22), it is derived that the variance of power is proportional to (and in the linked ref, equal to) the square of the mean power, i.e.

$$ var[ P(w)] = \bar P^2(w) $$

Distributions that follow the general relationship: $$var[ P(w)] =\alpha \bar P ^x(w)$$ are of the Tweedie family. In this case, the gamma distribution satisfies x=2.

---

A quick Python simulation with normally distributed noise where $$\mu=0, \sigma=6$$ corroborates this. The left plot is a histogram of variance computed over 4000 non-overlapping 1000-point time windows. Middle plot are the distributions of absolute Fourier amplitude, where each faint trace is the distribution at a single frequency (1000-point FFT). The right plot is the same, but for power (i.e. amplitude squared). The mean of those distributions (not shown) is indeed 0.036 (which is the signal variance over the number of FFT points, 6^2/1000). We see that the distributions are heavily skewed, and resemble the form of gamma distributions, but that's about as much as I can say.

Therefore, I'm venturing a guess that the noise distribution in the frequency domain follows the gamma distribution. I realize that this is without any formal derivation, and I would love to see a derivation of this as I am also interested in this problem.

---

Lastly, with reference to the above (Maximilian's) answer: please correct me if I'm misinterpreting your response, but the noise in the frequency domain cannot be Gaussian distributed with zero mean, right? Because the mean is defined as the mean power at that frequency, which is proportional to the variance of the signal in time domain. Sorry, I don't have enough points to comment directly below the response.

Edit: the Fourier coefficient themselves, i.e. real and imaginary components, are distributed normally with zero mean and standard deviation scaled to the total signal power, but not the signal magnitude.