3
$\begingroup$

Hi All: I'm trying to better understand the connection between variance of a time series and the integral of the spectral density over all frequencies. Rather than going through all of the relations, if one looks at the link below,

https://en.wikipedia.org/wiki/Spectral_density, the relations are shown there in the power spectral density section.

To summarize what is said there:

The power, $P$, is defined as the limit as $P=\lim_{T \to \infty} \frac{1}{2T} \int_{-T}^{T} |x(t)|^2$.

$S_{xx}(\omega)$ is defined as the limit as $S_{xx}(\omega)=\lim_{T \to \infty} E|\hat{x}_T(\omega)|^2$. So, I think of it as the expected instantaneous power at frequency $\omega$. I hope that's the way to think of it.

$\hat{x}_{T}(\omega)$ is defined as $\frac{1}{\sqrt T} \int_0^{T} x(t) e^{-i\omega t} dt$. So the fourier transform of $x_t$ divided by $\sqrt T$.

Finally, $\gamma_{0}$ is defined as the power contained in the total frequency band so $\int_{\omega} S_{xx} (\omega)$. My confusion is the following:

In the time domain-econometrics world, $\gamma_{0}$ is defined as $\mathrm{Var}(X_{t}) = E(X_{t} - \mu)^2$ where $\mu$ is the mean of $X_{t}$.

So, my confusion stems from the fact that when the variance is defined as power, there is no mean subtracted from $X_{t}$. Does that mean that $X_{t}$ has already been de-meaned when it's written as $X_t$. If not, then how can $\gamma_{0}$ be viewed as the variance when statistical variance is the (expected value of the deviation of a random variable from it's mean) squared yet the view of variance in the DSP world is a weighted (by frequency) combination of $E(X_{t}^2)$ over all frequencies. My only conclusion is that the mean is already being subtracted out or is assumed to be zero ? I basically don't understand how variance is defined in the DSP framework. Is it viewed as expected power rather than expected variability ? Thanks.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
5
$\begingroup$

Variance is never defined as power. For a wide-sense stationary random process $X(t)$ with zero mean

$$\mu_X=E\{X(t)\}=0\tag{1}$$

the variance of $X(t)$ equals its power.

The autocorrelation of $X(t)$ is defined by

$$R_X(\tau)=E\{X^*(t)X(t+\tau)\}\tag{2}$$

The power of $X(t)$ is

$$P_X=E\{|X(t)|^2\}=R_X(0)\tag{3}$$

The variance of $X(t)$ is

$$\sigma^2_X=E\{|X(t)-\mu_X|^2\}=E\{|X(t)|^2\}-|\mu_X|^2=R_X(0)-|\mu_X|^2\tag{4}$$

Clearly, if $\mu_X=0$, we have $\sigma^2_X=R_X(0)=P_X$.

Note that the autocovariance of $X(t)$ is defined by

$$C_X(\tau)=E\{[X(t)-\mu_X]^*[X(t+\tau)-\mu_X]\}=R_X(\tau)-|\mu_X|^2\tag{5}$$

Consequently,

$$\sigma^2_X=C_X(0)\tag{6}$$

But since the power spectral density $S_X(\omega)$ is the Fourier transform of $R_X(\tau)$, the integral over $S_X(\omega)$ equals the power of $X(t)$ and not its variance, unless $\mu_X=0$:

$$P_X=R_X(0)=\frac{1}{2\pi}\int_{-\infty}^{\infty}S_X(\omega)d\omega\tag{7}$$

Also take a look at this related answer.

For the difference between the definition of autocorrelation in statistics and in signal processing look here.

Improve this answer
$\endgroup$
8
  • $\begingroup$ Hi Matt: The issue must be notational then because, in statistical-time series and time series econometrics $R_{x}(0)$ is the autocovariance at lag zero which is the variance. ( no assumptions about zero mean ). Clearly, in your explanation ( which was quite nice by the way ), $R_{x}(0)$ is only the variance when $u_x = 0$. But the question kind of still remains then in that how does DSP deal with variance when the mean isn't zero ? Thanks for nice explanation. $\endgroup$
    – mark leeds
    Jan 14, 2018 at 18:30
  • $\begingroup$ Matt: The confusion basically stems from the fact that, in the non DSP world, $R_{x}({\tau})$ is defined as $E[(X_t+\tau - \mu)(X_t - \mu)$ so this is what caused my confusion. But, based on what you said, you have to demean the data in order to compute the variance using DSP techniques which is not difficult. Thanks. $\endgroup$
    – mark leeds
    Jan 14, 2018 at 18:41
  • $\begingroup$ @markleeds: Yes, then it must be notation. Variance is the autocovariance at lag $\tau=0$, not the autocorrelation at lag $\tau=0$, unless the mean is zero. What do you mean with how DSP deals with variance in the non-zero mean case? Just as it should, namely by considering the mean and not equating power and variance. $\endgroup$
    – Matt L.
    Jan 14, 2018 at 18:42
  • $\begingroup$ @markleeds: In DSP we usually use $R_X(\tau)$ to denote the autocorrelation, not the autocovariance (which we don't use very often). In your world, apparently $R_X(\tau)$ is the autocovariance. No big deal if that's all. $\endgroup$
    – Matt L.
    Jan 14, 2018 at 18:45
  • $\begingroup$ Matt: I mean ( no pun intended ) that, if you want to calculate the variance of your time series and you want to stay in DSP world, then you need to demean the data and then calculate the power by using the spectral density of the de-meaned series. Is that correct ? $\endgroup$
    – mark leeds
    Jan 14, 2018 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.