Effect of sampling a cont. stochastic process on the variance

Question

I am trying to understand the estimation of the power spectral density of a continuous time stochastic process from it's samples.

Consider a normal wide-sense stationary white noise process with average power $\sigma^2$. Its autocorrelation is a dirac-delta impulse of height $\sigma^2$. By the Wiener-Khinchin theorem, the power-spectral density of the process is the fourier transform of its autcorrelation function, i.e. constant $\sigma^2$.

Note that I'm avoiding to claim that any $x(t)$ in the process is a normally distributed random variable with variance $\sigma^2$. Apparently this notion is controversial.

In practice, one can estimate the power spectral density from a finite number of samples of the process from the DFT using the periodogram / Bartlett / Welch method.

Trying to generate these samples artificially in MATLAB, I noticed, that I need to scale the variance with the assumed sampling rate $f_s$ to get the right height in my periodogram.

x = sigma * sqrt(fs) * randn(N, 1)

Perhaps I just did not recognize it for what it was, but I never found this detail mentioned anywhere in the literature (except in the documentation on the band-limited white noise block of Simulink). This perhaps makes sense, because one usually samples a real process and does not try to generate these samples artificially, but perhaps someone could explain or point to a reference explaining the relationship between the variance of a set of samples from a stochastic process and the properties of the stochastic process itself. For example, Papoulis says that the autocorrelation of a samples process is the sampled version of the cont. time process' autocorrelation function, but nothing about variance is stated.

Answer 1

In order to have a finite variance, the white noise must be bandlimited prior to being sampled. If bandlimited to the Nyquist frequency or less, then the continuous time power spectral density will equal the discrete time power spectral density, and the discrete time samples will be "white" in that the power spectral density will be spread uniformly across the Nyquist range. If bandlimited to an amount greater than the sampling rate (but less than the analog bandwidth of the A/D converter) then the discrete time power spectral density will increase due to aliasing, predictably by the number of aliases that would fold in, up to the ADC bandwidth. In all cases where there is no further filtering of the signal the total power as given by $\sigma^2$ will be the same. We see this intuitively if we consider samples of a random process given by the following histogram; it does not matter what sampling rate we choose, the histogram with sufficient number of samples will remain the same! We recognize that the standard deviation, as given by $\sigma$, and therefore the variance or power as given by $\sigma^2$, remains unchanged. (As long as nothing in the process limited the bandwidth as used for the original histogram). We don't even have to sample uniformly; the only criteria is our sampling selection method is independent of the original sampling rate (if sampled) or the random process itself (if continuous time): If we select enough samples for statistical significance, we will get the same standard deviation.

The reason for the scaling is simple: the resulting periodogram is a power spectral density and for a discrete time white noise process, the power is distributed uniformly across the Nyquist frequency band. By scaling it by $f_s$ we can get the same power spectral density: If we doubled the sampling rate for example, we would need to double the variance to keep the density as power/Hz the same.