What does the covariance of a power spectral density between frequency points mean [duplicate]

Question

I am struggling to understand the meaning of the covariance of PSD (Power Spectral Density) between two frequency points ($f_1$ and $f_2$).

The covariance of a function $s(t)$ in time domain $t$ is a Fourier dual of the power spectral density and that is understandable. However, in some papers, the covariance of the power spectral density $S(f)$ is also attempted.

For example, in the case where it is assumed that $S(f)$ is a Gaussian process, the covariance of $S(f)$ is given by,

$$ K_F(f_1, f_2) = \frac{ S\left( \frac{f_1 + f_2}{2} \right) + S\left( \frac{f_1 - f_2}{2} \right) } {2} + j \frac{ S\left( \frac{f_1 + f_2}{2} \right) - S\left( \frac{f_1- f_2}{2} \right) } {2} $$ where $j = \sqrt{-1}$ and $S(f)$ is just the PSD. What does this $K_F$ mean physically?

EDIT: One such reference: Bayesian nonparametric spectral estimation.

Answer 1

See this related question which details further an intuition for covariance. From that we see that covariance is a correlation indicating a linear similarity between two functions.

To be clearer, the covariance alone and the power spectral density are not Fourier Transform pairs as suggested by the OP. It is the Autocovariance Function and the Power Spectral Density that are Fourier Transform pairs. And specifically, it is the Autocovariance Function of a time domain waveform that is generated by a wide sense stationary (WSS) process.

So what is the Autocovariance Function? Understanding that function first will help undestand what it means to compute that in the frequency domain instead of the time domain. The Autocovariance Function (also called the Auto-correlation Function) for a time domain function $x(t)$ is the covariance of the sequence with itself at a different delay $\tau$ in time, represented as $R_{xx}(t, t-\tau)$. Further, as a WSS process, such a result is only dependent on $\tau$ and can then be represented as $R_{xx}(\tau)$.

Therefore the autocovariance function, as computed in the time domain, indicates the "time memory" in the waveform, and how similar or correlated subsequent samples are to previous samples. "White noise" which is completely independent from sample to sample results in an autocovariance function that is an impulse in time, indicating no correlation between a current sample and all leading and trailing samples. The Fourier Transform of such a waveform is constant over all frequencies ("white"), indicative of the broad and high frequency content required to change instantly from one value to the next (in contrast to changing slowing in small increments, which has narrow frequency content). A narrow autocovariance indicates little time memory and independence from sample to sample, while a broader autocovariance indicates more time memory and dependence from sample to sample.

An example autocovariance (autocorrelation) function for a discrete time waveform that is independent from sample to sample is plotted below where the result is given as $r_{xx}(k)$ for discrete shift $k$.

So similarly we can compute the autocovariance function for a waveform that is in the frequency domain instead of the time domain. This would instead indicate "frequency memory" or how similar higher or lower frequencies are to a given frequency. A frequency waveform (not the power spectral density, but the waveform given as magnitude and phase versus frequency) that changes by very small amounts as we move from frequency to frequency would have a broader auto-covariance function. Likewise if the frequency values are completely independent, then we would have a narrow auto-covariance function. Just as "time memory" is associated with a narrower frequency content, "frequency memory" is associated with a narrow time content. The impulse response of a system and it's frequency response would be a good example: if we have a system (a filter) with a long "time memory", the frequency response of the filter is narrow (tight cut-off). Similarly if we have a filter with a long "frequency memory", it's impulse response (the time response) will be narrow. (short in time in long in frequency and vice-versa).

For the case of the autocovariance of the power spectral density specifically; such a computation would reveal the similarity in the power metric alone. This would be useful for determining the Resolution Bandwidth that was used in the spectral analysis, which unless a single test tone was utilized, would otherwise be difficult to determine from just observing the power spectral density. The resolution bandwidth is set by the duration of the time capture (for a stationary process) and any windowing function that may have been used. The width of the autocorrelation function in frequency would be directly proportional to the resolution bandwidth. More information on resolution bandwidth is given in this post.

As a quick demonstration of this I created a PSD for white Gaussian noise with a rectangular window and then again with a Kaiser window, and then with each PSD I computed the autocovariance as plotted below. The horizontal axis is the number of bins in the PSD (given the FFT used). The blue plot is the case for the rectangular window, showing the overall width to the first nulls as +/- 1 bin as expected. The orange plot shows the same case with the window used prior to computing the PSD, resulting in a wider autocovariance function consistent with the wider resolution bandwidth that window would create.