1
$\begingroup$

I have a time series that consists of noise and a signal, shown here windowed and Wiener filtered:

enter image description here

and the PSD of just the noise (used in filtering):

enter image description here

I want to find the variance of the noise using the PSD in order to estimate the significance of a measurement. I'm not super well-versed in signal processing, so I apologize in advance for any conceptual blunders! From digging around Wikipedia and StackExchange, I found that I can estimate the variance of a deterministic process like so:

$\sigma_{x} ^{2} = \int_{-\infty}^{\infty} S_{x}(f)df - \mu_{x}^{2}$

where $S_{x}(f)$ is the PSD and $\mu_{x}$ is the mean (side note: should there be a factor of $\frac{1}{\pi}$ or $\frac{1}{2\pi}$ in front of the integral?).

I then found this answer that confirmed that.

My question is: what modifications, if any, do I have to make to that formula to apply it to a stochastic process (the noise)? Also, the answer linked above specifies a stationary process- do I have to detrend the noise or can I assume that it is stationary?

$\endgroup$
1
$\begingroup$

The factor of $1/\pi$ or $1/(2\pi)$ in front depends on the convention chosen for $S_x$, i.e., one-sided vs two-sided PSD, and also expression of frequency in Hz vs rad/s. Therefore, it's a matter of using consistent conventions between the PSD and the integral. You can check that by seeing if the Wiener-Khinchin theorem is satisfied, https://en.wikipedia.org/wiki/Wiener%E2%80%93Khinchin_theorem.

The bigger issue is the evaluation of the PSD integral near $f=0$, which has to do also with your point about nonstationarity. You need to numerically evaluate the integral over the PSD, which is derived from the DFT of the original signal. The $\mu_x$ term removes the exact $f=0$ term in the DFT. However, there is a subtlety about the low frequency components: in practice, you rarely want to estimate the variance of the entire time-domain signal. I suspect that in applications, you would be more interested in the variance of the signal near the center of your window. Including the window edges will give a lower variance. By limiting the integral as

$\sigma_x(\tau) = \left(\int_{1/\tau}^{\infty} + \int_{-\infty}^{-1/\tau} \right) S_x(f) df$

you effectively detrend the data on timescales below $\tau$.

None of this depends on whether the signal is deterministic vs. stochastic - there's no way for the mathematics to tell the difference.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you! Is there any chance you could explain how limiting the integral detrends below $\tau$? $\endgroup$ – Petra Feb 27 at 21:43
  • 1
    $\begingroup$ when you limit the integral, you remove the low-frequency components of the power spectrum. this is roughly equivalent to applying a high-pass filter with corner frequency $1/(2\pi\tau)$. $\endgroup$ – Dave Kielpinski Feb 27 at 22:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.