Example of non-equivalence of the two PSD definitions

Question

According to the book Introduction to Spectral Analysis by P. Stoica and R. Moses, the power spectral density (PSD) $P(\omega)$ can either be defined as the discrete-time Fourier transform (DTFT) of the covariance sequence $r(k)$, i.e., \begin{align} P(\omega)=\sum_{k=-\infty}^{\infty}r(k)e^{-j\omega k}, \end{align} or alternatively as \begin{align} P(\omega)=\lim_{N\to \infty}E\Bigg\{\frac{1}{N}\Bigg|\sum_{n=0}^{N-1}x(n)e^{-j\omega n}\Bigg|^2\Bigg\}, \end{align} and these two definitions are equivalent under the assumption that \begin{align} \lim_{N\to \infty}\frac{1}{N}\sum_{k=-(N-1)}^{N-1}|k||r(k)|=0. \end{align} What is an example of a signal for which these two definitions yield different results, i.e., for which the above assumption does not hold?

Answer 1

Work in progress: wait till I am done before reading (and throwing brickbats!)

This question is difficult to answer without getting into a lot of details about basic signal analysis and Fourier transform theory.

Because of the way my brain works, I will discuss only real-valued continuous-time deterministic signals, and will get into the stochastic and discrete-time stuff later. The classical Fourier transform theory considers a finite-energy signal $x(t)$ and defines its Fourier transform as $$X(f) = \int_{-\infty}^\infty x(t) \exp(-j2\pi ft)\, \mathrm dt.\tag{1}$$ Note that $x(t)$ necessarily decays away to $0$ as $t \to \pm\infty$; absent this property, the signal cannot have finite energy as we have assumed.

The energy spectral density of $x(t)$ is defined to be $|X(f)|^2$ which happens to be the Fourier transform of the autocorrelation function $$r_x(t) = \int_{-\infty}^\infty x(\tau)x(\tau+t)\, \mathrm d\tau.\tag{2}$$ Note that $r_x(0)$ equals the finite energy of the signal.

Now, all of this is fine and dandy but it doesn't work power signals which are signals that have infinite energy but finite (average) power, that is, $$\mathcal P_T = \frac{1}{2T}\int_{-T}^T |x(t)|^2\, \mathrm dt, \tag{3}$$ which will be recognized as the average power delivered by $x(t)$ (into a $1\Omega$ resistor) over the $2T$-second interval $[-T,T]$, approaches a finite limit $\bar{\mathcal P}$ as $T \to \infty$. The power signals that everyone is familiar with are periodic signals (generally represented by Fourier series) and to accommodate these, we include Dirac deltas or impulses into our theory and pretend that the right side of Eq. $(1)$ "converges" to an impulse or a sum of impulses.

The Fourier transform of a periodic signal with fundamental frequency $f_0$ consists of impulses at the harmonics $nf_0, n \in \mathbb Z$, of the fundamental frequency. Specifically, if the periodic signal $x(t)$ has Fourier series $\displaystyle\sum_{n=-\infty}^\infty c_n \exp(j2\pi nf_0 t)$, then the Fourier transform $X(f)$ of $x(t)$ is defined to be $$X(f) = \sum_{n=-\infty}^\infty c_n \delta(f-nf_0),$$ where $\delta(\cdot)$ denotes an impulse or Dirac delta.

This has become second nature to us by now and we even incorporate impulses into tables of Fourier transforms and the like, and get into arguments about whether the Fourier transform of an impulse train is another impulse train or not. Note that the power spectral density of $x(t)$ is defined to be $S_x(f) = |X(f)|^2$ and for the case of periodic $x(t)$, we have that $$x(t) = \sum_{n=-\infty}^\infty c_n \exp(j2\pi nf_0 t) \implies X(f) = \sum_{n=-\infty}^\infty c_n \delta(f-nf_0)\\ ~\text{and}~ \\|X(f)|^2 = \sum_{n=-\infty}^\infty |c_n|^2 \delta(f-nf_0).$$ If you choose to "square" the sum $X(f)$ to arrive at $|X(f)|^2$, please remember that $\delta(f-nf_0)\delta(f-mf_0)$ is $\delta(f-nf_0)$ if $n$ equals $m$ and $0$ if $n\neq m$. The autocorrelation function of $x(t)$ is the inverse Fourier transform of the power spectral density and is thus $$r_x(t) = \sum_{n=-\infty}^\infty |c_n|^2 \exp(j2\pi nf_0 t)$$ which is also a periodic function.

---

But what if $x(t)$ is a power signal but is not a periodic signal and so we can't brute-force use Eq. $(1)$ and hand-wave our way to a power spectral density consisting of impulses only? Well, one way to proceed is to re-consider Eq. $(3)$ and note that $\mathcal P_T$ can be expressed as $$\mathcal P_T = \frac{1}{2T}\int_{-T}^T |x_T(t)|^2\, \mathrm dt = \frac{1}{2T}\int_{-\infty}^\infty |x_T(t)|^2\, \mathrm dt\tag{4}$$ where $x_T(t)$ defined as $$x_T(t) = \begin{cases}x(t), & -T \leq t \leq T,\\0, &\text{otherwise,}\end{cases}.\tag{5}$$ Note that $x_T(t)$ is a finite-energy signal no matter how large $T$ is (as long as $T$ is finite) and thus it has a Fourier transform $X_T(f)$. Note also that $x_T(t)$ has finite support $[-T,T]$ while the support of $X_T(f)$ is the entire frequency axis $-\infty < f < \infty$. Next, note that Parseval's relation lets us re-write Eq.$(4)$ as $$\mathcal P_T = \frac{1}{2T}\int_{-\infty}^\infty |X_T(f)|^2\, \mathrm df\tag{6}$$ leading to \begin{align} \bar{\mathcal P} &= \lim_{T \to \infty}\mathcal P_T\\ &= \lim_{T \to \infty} \frac{1}{2T}\int_{-\infty}^\infty |X_T(f)|^2\, \mathrm df\\ &= \int_{-\infty}^\infty \lim_{T \to \infty} \frac{1}{2T}|X_T(f)|^2\, \mathrm df &{\scriptstyle{\text{since}~\bar{\mathcal P}~\text{is finite by assumption.}}}\tag{7} \end{align} But the average power is just the area under the power spectral density curve, that is, $$\bar{\mathcal P} = \int_{-\infty}^\infty S_x(f)\, \mathrm df$$ and thus the power spectral density of this finite-power signal is defined as \begin{align}S_x(f) &= \lim_{T \to \infty} \frac{1}{2T}|X_T(f)|^2\tag{8}\\ &= \lim_{T \to \infty} \frac{1}{2T}\left|\int_{-\infty}^\infty x_T(t)\exp(-j2\pi ft) \,\mathrm dt\right|^2\tag{9}\\ &= \lim_{T \to \infty} \frac{1}{2T}\left|\int_{-T}^T x(t)\exp(-j2\pi ft) \,\mathrm dt\right|^2\tag{10} \end{align} which (except for the expectation operator -- not needed because everything is deterministic here) looks a lot like the second definition of the power spectral density in the OP's question.

The autocorrelation function of $x(t)$, which is the inverse Fourier transform of the power spectral density $S_x(f)$ in Eq. $(8)$ is \begin{align}r_x(t) &= \lim_{T \to \infty} \frac{1}{2T}\int_{-\infty}^\infty x_T(\tau)x_T(t+\tau) \,\mathrm d\tau\\ &= \lim_{T \to \infty} \frac{1}{2T}\int_{-T}^T x_T(\tau)x_T(t+\tau) \,\mathrm d\tau &{\scriptstyle{\text{since}~x_T(\tau)=0~\text{when }~|\tau|>T}}\\ &= \lim_{T \to \infty} \frac{1}{2T}r_{x_T}(t).\tag{11} \end{align} Note that $r_{x_T}(t)$ has support $[-2T,2T]$ but is a poor approximation to $r_x(t)$ when $|t| > T$ because the overlap between the support $[-T, T]$ of $x_T(\tau)$ and the support of $x_T(t+\tau)$ is small.

---

OK, but what about random processes which is what the OP was asking about? Well, the problem is our model for a random process is a collection of random variables $\{\mathscr X(t)\colon t\in \mathbb R\}$ whereas what we observe with our spectrum analyzers and oscilloscopes is a finite segment, say, $x_T(t)$ of one sample path $x(t)$ among the many sample paths of the process. So, for each $t \in [-T,T]$, we know the value that the random variable $\mathscr X(t)$ took on for the one outcome $\omega$ in the underlying sample space $\Omega$ but it is quite difficult to get much information about even the basic properties of $x_T(t)$ (e.g. is $x_T(t)$ a continuous function? does it have a Fourier transform? etc.) from the bald description of the random process as "a collection of random variables". So, let's assume that the random process is wide-sense-stationary and that it satisfies various ergodic theorems that allow for estimation of the power spectral density and the autocorrelation function of the process from the segment $x_T(t)$ (and its Fourier transform $X_T(f)$) which is the only part of the single sample path $x(t)$ available to us. The Wiener-Khinchin theorem says the power spectral density $S_{\mathscr X}(f)$ of the process $\{\mathscr{X}(t)\}$ is given by \begin{align} S_{\mathscr X}(f) &= E\left[\lim_{T \to \infty} \frac{1}{2T}\left|\int_{-T}^T \mathscr{X}(t)\exp(-j2\pi ft) \,\mathrm dt\right|^2\right]\tag{12} \end{align} and we can approximate this by $$\frac{1}{2T}\left|\int_{-T}^T x(t)\exp(-j2\pi ft) \,\mathrm dt\right|^2 = \frac{1}{2T}\left|\int_{-T}^T x_T(t)\exp(-j2\pi ft) \,\mathrm dt\right|^2.$$ It would appear reasonable that we should be able to express the power spectral density $S_{\mathscr X}(f)$ as the Fourier transform of the process autocorrelation function $R_{\mathscr X}(t)$ which we can approximate by $r_{x_T}(t)$??:

$$S_{\mathscr X}(f) \approx \frac{1}{2T}\int_{-\infty}^\infty r_{x_T}(t)\exp(-j2\pi ft) \, \mathrm dt?? \tag{13}$$ Well, $r_{x_T}(t)$ is known only for $|t| \leq 2T$ and might not be all that great an estimate of the value of $R_{\mathscr X}(t)$ when $|t| \geq T$. So, the Wiener-Khinchin theorem also says that the