10
$\begingroup$

I am currently struggling on the book "System Identification: Theory for the User" by Lenart Ljung (freely available here) about the definition of the spectrum (around page 33). As I have seen elsewhere, it is defined as the Fourier transform of a fancy autocorrelation function. This gave rise to many questions and I would like to know when it is true, with what autocorrelation (more on that later) and when we need to define the spectrum this way.

First of all, my vision of the Fourier transform is pretty straightforward. Whether we use the canonical continuous version: $$X(\omega) = \int_{-\infty}^{+\infty}x(t)e^{-i\omega t}dt \tag{1}\label{eq1}$$ or its discrete counterpart: $$X_n = \sum_{k=0}^{N-1}x_k e^{-i\frac{2k\pi}{N}n} \tag{2}\label{eq2}$$

is that the power spectral density (PSD) is $$PSD(w) =|X(\omega)|^{2} \tag{3}\label{eq3}$$

resp.: $$PSD(\frac{2\pi n}{N}) = |X_n|^{2} \tag{4}\label{eq4}$$

based on the fact that $X(\omega)$ (resp. $X_n$) is the squared norm of the projection of the signal $x$ on the function $t\mapsto e^{i\omega t}$ (resp. the sequence $k\mapsto e^{i\frac{2\pi n}{N} k}$) seen as a vector in a vector field owning the following dot product:

$$\langle f\vert g \rangle = \int_{-\infty}^{+\infty} f(t)\overline{g(t)}dt \tag{5}\label{eq5}$$

resp. $$\langle u_n\vert v_n \rangle = \sum_{k=0}^{+\infty} u_k \overline{v_k} \tag{6}\label{eq6}$$

First problem

My first problem lies within the very definition of the autocorrelation. On one hand you find in many sources that, where $x(t)$ (resp. $x_t$) is a stationary random variable (i.e., its probabilistic do not depend on time) you have for the continuous case: $$\forall \tau\in\mathbb{R} \quad R_x(\tau) = E(x(t)x(t+\tau)) \tag{7}\label{eq7}$$ and for the discrete case $$\forall \tau\in\mathbb{Z} \quad R_x(\tau) = E(x_t x_{t+\tau}) \tag{8}\label{eq8}$$

or, as it is defined for instance here:

$$\forall \tau\in\mathbb{R} \quad R_x(\tau) = \int_{-\infty}^{+\infty}x(t)x(t+\tau) dt \tag{9}\label{eq9}$$ and for the discrete case $$\forall \tau\in\mathbb{Z} \quad R_x(\tau) = \sum_{-\infty}^{+\infty}x_t x_{t+\tau} \tag{10}\label{eq10}$$

Well ... Which is it? I'm not exactly a probability/statistic genius, but, for instance, for the continuous case, for \ref{eq7} to be consistent with \ref{eq9} we should have:

$$\forall \tau\in\mathbb{R} \quad R_x(\tau) = \int_{-\infty}^{+\infty}x(t)x(t+\tau) f_{x(t)}(x(t)) f_{x(t+\tau)}(x(t+\tau))dt \tag{11}\label{eq11}$$ with $f_{a}(a)$ being the probability density function of the random variable $a$ (in subscript) evaluated on the realization of this variable (the argument between the parenthesis), which I also noted $a$.

Second problem

This problem is very simple. When are equation \ref{eq3} and \ref{eq4} true? With what signal? what autocorrelation? When are they not true? When are we forced to define the spectral density as the Fourier transform of the autocovariance because \ref{eq3} and \ref{eq4} do not hold anymore? What sense does it make then?

I knew a least elegant, but very similar, version of this dsp.stackexchange answer and it seems ok, but when I tried to replicate the proof in the discrete context, so many questions arrived like: is my signal of finite length? If the signal is of finite length, are we considering it periodised version or not? What is the discrete Fourier transform for a signal of infinite length?

In the book there seems to be an impressive number of cases. Is the signal of finite length? Is the signal periodic? What if we consider the periodised version of the finite length signal? And so on...

Can somebody give me a clear overview of this mess? Is there a general framework that resists these infinite variations?

$\endgroup$
3
  • 5
    $\begingroup$ Thank you for numbering your equations. When things get complicated that makes things oh so much easier. $\endgroup$
    – TimWescott
    Mar 24, 2023 at 18:40
  • 2
    $\begingroup$ The Wiener-Khinchin theorem should probably be mentioned. $\endgroup$ Mar 25, 2023 at 19:41
  • $\begingroup$ @VladimirFГероямслава yes thank you. I found a nice proof here. I'm working on a full response for my question and this theorem was very useful! $\endgroup$
    – NokiYola
    Mar 25, 2023 at 20:36

4 Answers 4

9
$\begingroup$

The FIRST PROBLEM you ask about is how does

$$R_x(\tau) = E[x(t)x(t+\tau)] $$

equal

$$R_x(\tau) = \lim_{T\rightarrow \infty} \frac{1}{2T} \int_{-T}^{+T} x(t)x(t+\tau) dt$$

because (7) requires knowledge of the statistics (the probability density function (PDF)) of $x$, but (9) does not.

This "equivalence" is made because of the assumption that $x$ is ergodic.

There is a longer explanation here, but an interpretation of what ergodicity means is that we can replace

$$ E[x(t)] = \int_{-\infty}^{+\infty} u f_x(u) du$$

with

$$ \lim_{T\rightarrow \infty} \frac{1}{2T} \int_{-T}^{+T} x(t) dt$$

or we can replace the ensemble averages (using the probabilty density function $f_x$ ) with the time averages.

$\endgroup$
2
  • 1
    $\begingroup$ Thank you for your answer. I'll have a look and I'll try to demonstrate the equality of both autocovariance definitions. $\endgroup$
    – NokiYola
    Mar 24, 2023 at 21:52
  • 1
    $\begingroup$ Can I edit your answer, Peter? Revert me if you must. $\endgroup$ Mar 24, 2023 at 23:00
9
$\begingroup$

To add to Peter K.'s answer:

The correct defition of the autocorrelation is $ R_x(\tau) = E[x(t)x(t+\tau)]$.

If the process is stationary, that simplifes a little to $ R_x(\tau) = E[x(0) x(\tau)]$.

If the process is also ergodic, then it can be computed as $\lim_{T\rightarrow \infty} \frac{1}{2T} \int_{-T}^{+T} x(t)x(t+\tau) dt$

To help understand the equivalence (up to a point) of those two very different definitions, consider the expectation of a random value. The definition is $E[X]=\int f_X(x) dx$. Now, under certain conditions and in a certain sense, we can replace the expectation by a very different thing, a sample average: $\lim_{N\rightarrow \infty} \frac{1}{N}\sum_{i=1}^N X_i$. Actually, what we are saying is that an estimator (the sample average) converges (in some sense) to the true expectation.

Going back to the autocorrelation, imagine we have a discrete-time stochastic process. Then we could attempt to replace the expectation by a sample average (in this case, averaging over "time")

$$ E[x(t)x(t+\tau)] \approx \frac{1}{N}[x(1)x(1+\tau) + x(2)x(2+\tau) + \cdots x(N)x(N+\tau)] $$

In the case of a continuous-time process, we replace the sum by an integral.

Under which conditions this will work? To start with, we need the process to be stationary. But we need more than that. What more? Well, that's a little complicated. Technical details aside, we'll call those processes "ergodic", and we'll hope that most stationary processes are ergodic. In that case, we are justified in substituting the true expectation by a time-average.

Regarding your second problem, there is a similar confusion between (fixed) parameters of a random process and estimators, which ar functions of realizations of the random process (and hence, random). Your definition (3) of the PSD applies only to deterministic signals. But if $x(t)$ is a stochastic process, then your $PSD(w)$ would be a random value - that cannot be right. The PSD of a random process is defined as the Fourier transform of the autocorrelation function (and hence it's itself a fixed-deterministic function).

What you define in $(3)$ is actually the periodogram of the random process, which is itself a random value, and which could be regarded as an estimator of the PSD. Unfortunalely, that naive estimator is not particularly good (it's not consistent). Other methods are preferred.

$\endgroup$
7
$\begingroup$

The magnitude-square $\Big| X(\omega) \Big|^2$ of the Fourier Transform of energy signals is the energy spectral density.

For power signals, you gotta do a little bit with normalization, so that integrals do not necessarily blow up, but $\Big| X(\omega) \Big|^2$ is sorta proportional to the power spectrum.

This means for finite power signals that are ergodic in every sense, any expectation value:

$$ E[g\big\{x(t)\big\}] = \int_{-\infty}^{+\infty} g\big\{u\big\} p_x(u) \ \mathrm{d}u$$

can be equivalent to

$$ \lim_{T\rightarrow \infty} \frac{1}{2T} \int_{-T}^{+T} g\big\{x(t)\big\} \ \mathrm{d}t$$

for essentially any decently defined function $g\{\cdot\}$. There may be details that I forgot 4 decades ago.

Now start applying that to the autocorrelation function.

$$\begin{align} E[x(t)\, x(t+\tau)] &= \int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty} u v \ p_{x(t)x(t+\tau)}(u, v) \ \mathrm{d}u\mathrm{d}v \\ &= \lim_{T\rightarrow \infty} \frac{1}{2T} \int_{-T}^{+T} x(t)\, x(t+\tau) \ \mathrm{d}t \end{align}$$

$\endgroup$
4
  • 3
    $\begingroup$ And then you need to be very careful to say power spectral density. $\endgroup$
    – TimWescott
    Mar 24, 2023 at 18:42
  • 3
    $\begingroup$ And interesting to me that the units of the power spectral density itself is energy in the frequency domain: Power/Hz is energy $\endgroup$ Mar 24, 2023 at 20:05
  • $\begingroup$ Thank you for your answer. Actually your hyperlink echoes to something in the book. For discrete signals with instants numbered in $\mathbb{N}^{\star}$ the power spectral density is defined as $R_s(\tau) = \lim_{N\to\infty} \frac{1}{N}\sum_{t=1}^{N} E(s(t)s(t-\tau)))$ $\endgroup$
    – NokiYola
    Mar 24, 2023 at 21:59
  • 1
    $\begingroup$ I might be asking something stupid but what is the point in treating infinite time extension systems while you won't see their end? $\endgroup$
    – NokiYola
    Mar 24, 2023 at 22:01
4
$\begingroup$

I want to thank everybody for their answers and comments : each one of them actually gave me a new insight to the problem.

This is a very long post but I tried to make a comprehensive list of all the cases I learned while studying the answers with the most mathematical justification I could. Feel free to make any comment or suggestion.

Context and basic definitions :

In what follows :

  • E[.] is the probability expectancy operator
  • Finite length signals are starting at 0 and are of length T (continuous) and N (discrete)
  • $\mathcal{F}$ is the right Fourier transformation adapted to the situation
  • $\tau$ is the time difference that defines the autocorrelation. Wether it's real of integer is obvious in the context.
  • Signals with integer subscripts like $h_i$ are discrete signals, signals with arguments like $h(t)$ are in continuous time signals
  • Signals in uppercase are the results of the Fourier transformation of the lower case signal. Example : $H = \mathcal{F}(h)$

There seems to be three major dichotomies that superimpose on top of one another to answer the question at hand :

  1. Finite energy signals VS power signals
  2. Discrete signals VS continuous signals
  3. stochastic signals VS deterministic signals

I did not cover every cases but I covered what I believe to be the most commons.

About finite energy and power signals

Finite energy signals are signals x that verify :

$$\int_{-\infty}^{+\infty}\vert x(t)\vert^{2}dt <+\infty \quad \text{(resp. } \sum_{k=-\infty}^{+\infty}\vert x(k T_{s})\vert^{2} <+\infty )$$ The previous integral (resp. series) is called the energy.

Also, surprisingly, if you have finite energy, you actually don't need to verify :

$$\int_{-\infty}^{+\infty}\vert x(t)\vert dt <+\infty \quad \text{(resp. }\sum_{k=-\infty}^{+\infty}\vert x(kT{s})\vert <+\infty )$$ to take a Fourier transform, thanks to Plancherel's theorem (not Parseval's formula, the other Plancherel's theorem. A proof of both here and here)

In the finite energy context the Energy Spectral Density (ESD) is then naturally defined as :

$$ESD(x)(\omega) = \vert \mathcal{F}(x)(\omega)\vert^{2} $$

with $\mathcal{F}$ being the Fourier transformation that applies in the context (see below for more details on the various contexts)

It is called Energy spectral density because the sum/integral of $\vert \mathcal{F}(x)(\omega)\vert^{2}$ is called the energy.

Sometimes this sum/integral does not exist and to access frequency information we need to normalize/average with respect to time in order to evaluate a finite quantity.

=> This division time will lead us to talk about Power Spectral Density (PSD) instead of ESD.

That kind of signals are then also called power signals. They are treated at the end as a special case.

Autocorrelation definitions :

Finite energy signals are usually considered deterministic. Therefore their autocorrelation is usually defined as : $$R_{xx}(\tau) = \int_{-\infty}^{+\infty} x(t)\overline{x(t-\tau)}dt$$ resp. $$R_{xx}(\tau) = \sum_{k=-\infty}^{+\infty} x_k\overline{x_{k-\tau}}dt$$

with some adaptations for finite length signals (I'll give details when needed).

In a stochastic context, the autocorrelation is defined as :

$$R_{xx}(\tau) = E[x(t)\overline{x(t-\tau)}]$$

Now it's going to be a bit repetitive but let's show that this relationship $R_{xx}(\tau) = \mathcal{F}^{-1}(PSD)$ is true no matter the paradigm, starting with the finite energy signals.

Case finite energy - continuous time - continuous frequencies (i.e infinite signal):

in that context we have $X(\omega) = \int_{-\infty}^{+\infty}x(t) e^{-i\omega t}dt$ and $x(t) = \int_{-\infty}^{+\infty}X(\omega) e^{i\omega t}d\omega$

So we have : $$ \begin{split} R_{xx}(\tau) &= \int_{-\infty}^{+\infty}x(t)\overline{x(t-\tau)}dt\\ &=\int_{-\infty}^{+\infty} [\int_{-\infty}^{+\infty}X(\omega)e^{i\omega t}d\omega]\overline{x(t-\tau)}dt\\ &=\int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty}X(\omega)e^{i\omega\tau}\overline{x(t-\tau)}e^{i\omega (t-\tau)}d\omega dt\\ \end{split} $$ Now in order to invert the two integrals we need first to show that $I(\omega) = \int_{-\infty}^{+\infty}X(\omega)e^{i\omega\tau}\overline{x(t-\tau)}e^{i\omega (t-\tau)}dt = X(\omega)\overline{X(\omega)}e^{i\omega\tau}$ exists.

It does, it's a constant multiplying a reverse Fourier transform that exists by Plancherel's theorem.

Now we also need to show that $\int_{-\infty}^{+\infty}\vert I(\omega)\vert dw = \int_{-\infty}^{+\infty} \vert X(\omega)\vert^{2}$ exists.

It does thanks to Plancherel's theorem (Parseval's formula version) and the fact that we assumed $x$ to be square integrable we know that $X(\omega)$ also is. Therefore we get :

$$\begin{split} R_{xx}(\tau) &= \int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty} X(\omega)e^{i\omega\tau}\overline{x(t-\tau)}e^{i\omega (t-\tau)}dt d\omega\\ &=\int_{-\infty}^{+\infty} \vert X(\omega)\vert^{2}e^{i\omega\tau}d\omega \end{split}$$

Which is equivalent to $R_{xx} = \mathcal{F}^{-1}(ESD)$

Case finite energy - continuous time - discrete frequencies (i.e finite signal):

This one was tricky. I couldn't go around considering the periodised signal, $$x_p(t) = \sum_{k=-\infty}^{+\infty} x(t-kT)$$

It makes sense since the spectrum coefficients in that context are the Fourier coefficients of the periodised signal.

Also the definition of the autocorrelation in that case is :

$$R_{xx}(\tau) = \frac{1}{T}\int_{T}x_p(t)x_p(t-\tau)dt $$

In that context we have $$X_k = \frac{1}{T}\int_{0}^{T}x(t)e^{-i\frac{2\pi}{T}kt}dt = \frac{1}{T}\int_{T}x_p(t)e^{-i\frac{2\pi}{T}kt}dt$$ and $$x(t) = \sum_{k=-\infty}^{+\infty} X_k e^{i\frac{2\pi}{T}k t}\quad \forall t\in [0, T]$$ and for the perdiodised function : $$x_p(t) = \sum_{k=-\infty}^{+\infty} X_k e^{i\frac{2\pi}{T}k t} \quad \forall t\in \mathbb{R}$$.

Note the $\frac{1}{T}$ factor in the Fourier coefficient. This will allow us to talk about PSD (Power Spectral Density) instead of ESD (Energy Spectral Density). Note also how $\int_{-\infty}^{+\infty}\vert x_p(t)\vert^{2}dt$ does not converge due to periodicity.

$$ \begin{split} R_{xx}(\tau) &= \frac{1}{T}\int_{T} x(t)\overline{x(t-\tau)}dt\\ &= \frac{1}{T}\int_{T} \sum_{k=-\infty}^{+\infty} X_k e^{i\frac{2\pi}{T}k t}\overline{x(t-\tau)}dt \\ &=\frac{1}{T}\int_{T} \sum_{k=-\infty}^{+\infty} X_k e^{i\frac{2\pi}{T}k \tau}\overline{x(t-\tau)}e^{i\frac{2\pi}{T}k (t-\tau)}dt \end{split} $$

in order to switch the integral and the sum we first need to show that $\frac{1}{T}\int_{T} X_k e^{i\frac{2\pi}{T}k \tau}\overline{x(t-\tau)}e^{i\frac{2\pi}{T}k (t-\tau)}dt = X_k e^{i\frac{2\pi}{T}k \tau}\frac{1}{T}\int_{T}\overline{x(t-\tau)}e^{i\frac{2\pi}{T}k (t-\tau)}dt$ exists.

It does because it is a classical integral on a interval. To be specific if $x$ is piecewise continuous on $[0, T[$ then $x_p$ which is T-periodic and coincides with $x$ on $[0,T[$ has to be piecewise continuous by periodicity. I guess it's trickier if $x$ doesn't have a finite support but luckily we're not in that case.

We also need to show that $X_k e^{i\frac{2\pi}{T}k \tau}\frac{1}{T}\int_{T}\overline{x(t-\tau)}e^{i\frac{2\pi}{T}k (t-\tau)}dt= \vert X_k\vert^{2}e^{i\frac{2\pi}{T}k \tau} $ is summable.

It is since $\vert \vert X_k\vert^{2}e^{i\frac{2\pi}{T}k}\vert = \vert X_k\vert^{2} $ which is summable via Parseval's forumula applied on the Fourier coefficients of $x_p$.

Then we can conclude :

$$ \begin{split} R_{xx}(\tau) &= \sum_{k=-\infty}^{+\infty} \frac{1}{T}\int_{T} X_k e^{i\frac{2\pi}{T}k \tau}\overline{x(t-\tau)}e^{i\frac{2\pi}{T}k (t-\tau)}dt\\ &= \sum_{k=-\infty}^{+\infty} X_k e^{i\frac{2\pi}{T}k \tau}\frac{1}{T}\int_{T}\overline{x(t-\tau)}e^{i\frac{2\pi}{T}k (t-\tau)}dt\\ &=\sum_{k=-\infty}^{+\infty} X_k e^{i\frac{2\pi}{T}k \tau}\overline{X_k}\\ &=\sum_{k=-\infty}^{+\infty} \vert X_k\vert^{2} e^{i\frac{2\pi}{T}k \tau} \end{split} $$

Which is equivalent in that context to $R_{xx} = \mathcal{F}^{-1}(PSD)$

Case finite energy - discrete time - continuous frequencies (i.e infinite signal):

In that context $X_{2\pi}(\omega) = \sum_{-\infty}^{+\infty} x_k e^{-i\omega k}$ and $x_k = \frac{1}{2\pi}\int_{2\pi}X_{2\pi}(w)e^{i\omega k}$ So we have : $$ \begin{split} R_{xx}(\tau) &= \sum_{k=-\infty}^{+\infty}x_k \overline{x_{k-\tau}} \\ &= \sum_{-\infty}^{+\infty}[\frac{1}{2\pi}\int_{2\pi}X_{2\pi}(w)e^{i\omega k}d\omega] \overline{x_{k-\tau}}\\ &= \frac{1}{2\pi}\sum_{k=-\infty}^{+\infty} \int_{2\pi}X_{2\pi}(w)e^{i\omega k} \overline{x_{k-\tau}}d\omega\\ &= \frac{1}{2\pi}\sum_{k=-\infty}^{+\infty} \int_{2\pi}X_{2\pi}(w)e^{i\omega \tau} \overline{x_{k-\tau}}e^{i\omega (k-\tau)}d\omega\\ \end{split} $$ In order to exchange the integral and the sum we need first to show that $I(\omega) = \sum_{k=-\infty}^{+\infty}X_{2\pi}(w)e^{i\omega \tau} \overline{x_{k-\tau}}e^{i\omega (k-\tau)}= X_{2\pi}(w)e^{i\omega \tau}\sum_{k=-\infty}^{+\infty} \overline{x_{k-\tau}}e^{i\omega (k-\tau)}$ exists.

It does, it is a constant multiplying a Fourier transform (DTFT in that context) which exists because we assumed the signal to be square summable.

Now we need to show that $X_{2\pi}(w)e^{i\omega \tau}\sum_{k=-\infty}^{+\infty} \overline{x_{k-\tau}}e^{i\omega (k-\tau)} = X_{2\pi}(w)e^{i\omega \tau}\overline{X_{2\pi}(w)} = \vert X_{2\pi}(w)\vert^{2}e^{i\omega \tau}$ is piecewise continuous.

I actually don't know how to do that but I guess if we recognize $X(\omega)$ as a Fourier series it comes easy. We can therefore intervert the sum and the integral. We get :

$$\begin{split} R_{xx}(\tau) &= \frac{1}{2\pi}\int_{2\pi}X_{2\pi}(\omega)e^{i\omega \tau}\sum_{k=-\infty}^{+\infty} \overline{x_{k-\tau}}e^{i\omega (k-\tau)}\\ &= \int_{2\pi}X_{2\pi}(w)e^{i\omega \tau}\overline{X_{2\pi}(w)} \\ &= \int_{2\pi}\vert X_{2\pi}(w)\vert^{2}e^{i\omega \tau}\\ \end{split}$$

Which is equivalent in that context to $R_{xx}(\tau) = \mathcal{F}^{-1}(ESD)$

Case finite energy - discrete time - discrete frequencies (i.e finite signal):

In that context we also need to periodise the signal with the periodised sequence $\tilde{x}_k$ being defined by $\forall s\in \mathbb{Z}\quad \tilde{x}_{s+N}=\tilde{x}_{s}$ and $\forall 0\leq t\leq N-1 \quad\tilde{x}_t=x_t$. The sequence $(\tilde{x}_t)_{t\in \mathbb{Z}}$ is then N-periodic.

In that context the Fourier transform is the DFT defined by :

$$X_k = \sum_{t=0}^{N-1}x_t e^{-i\frac{2\pi}{N}kt} = \sum_{N}\tilde{x}_t e^{-i\frac{2\pi}{N}kt}$$

also :

$$\forall 0\leq t \leq N-1 \quad x_t = \frac{1}{N}\sum_{k=0}^{N-1}X_k e^{i\frac{2\pi}{N}kt}$$

And finally :

$$\forall t \in \mathbb{Z} \quad \tilde{x}_t = \frac{1}{N}\sum_{k=0}^{N-1}X_k e^{i\frac{2\pi}{N}kt}$$

Note the $\frac{1}{N}$ factor in the Fourier coefficient. This will allow us to talk about PSD (Power Spectral Density) instead of ESD (Energy Spectral Density). Note also how $\sum_{t=-\infty}^{+\infty}\vert \tilde{x}_t(t)\vert^{2}$ does not converge due to periodicity.

and the convolution on the periodised N-periodic sequences is called the circular convolution. For the autocorrelation we get :

$$R_{xx}(\tau) = \sum_{t=0}^{N-1}x_t\overline{\tilde{x}_{t-\tau}} = \sum_N \tilde{x}_{t}\overline{\tilde{x}_{t-\tau}}$$

Let us start the proof :

$$ \begin{split} R_{xx}(\tau) &= \sum_{t=0}^{N-1}x_t\overline{\tilde{x}_{t-\tau}}\\ &= \sum_{t=0}^{N-1} [\frac{1}{N}\sum_{k=0}^{N-1}X_ke^{i\frac{2\pi}{T}kt}] \overline{\tilde{x}_{t-\tau}}\\ &=\frac{1}{N}\sum_{k=0}^{N-1}\sum_{t=0}^{N-1}X_ke^{i\frac{2\pi}{T}kt} \overline{\tilde{x}_{t-\tau}}\\ &=\frac{1}{N}\sum_{k=0}^{N-1}X_ke^{i\frac{2\pi}{T}k\tau} \sum_{t=0}^{N-1}\overline{\tilde{x}_{t-\tau}}e^{i\frac{2\pi}{T}kt}\\ &=\frac{1}{N}\sum_{k=0}^{N-1}X_ke^{i\frac{2\pi}{T}k\tau} \overline{X_k}\\ &=\frac{1}{N}\sum_{k=0}^{N-1}\vert X_k\vert^{2} e^{i\frac{2\pi}{T}k\tau}\\ \end{split} $$

Which is equivalent in that context to $R_{xx} = \mathcal{F}^{-1}(PSD)$

Case of discrete/continuous power signals (not square integrable/summable)

I imagine this context exists to study very long signals.

For power signals, we don't assume that $\int_{-\infty}^{+\infty}\vert f(t)\vert^{2}dt <+\infty $ (resp. $\sum_{k=-\infty}^{+\infty}\vert f(k T_{s})\vert^{2} <+\infty $ ). In fact they might have infinite energy, i.e : $$\lim_{a,b \to +\infty}\int_{-a}^{b} \vert x(t)\vert^{2}dt = +\infty$$ resp. $$\lim_{a,b \to +\infty}\sum_{k=-a}^{b} \vert x(kT_s)\vert^{2} = +\infty$$ which is very unfortunate, because then we can't apply the Fourier transform.

To get around that issue we consider the signals to be stochastic and, when the limit exists, we define the spectrum as :

$$S_{xx}(\omega) = \lim_{T\to +\infty}\frac{1}{T}E[\vert X_T(\omega)\vert^{2}]$$ with $$X_T(\omega) = \int_{-\frac{T}{2}}^{\frac{T}{2}} x(t)e^{i\omega t} dt$$

Note the normalized limit which suggests a hidden ergodic process assumption (however I am not sure of that).

Also note the $\frac{1}{T}$ normalization factor. As expected we talk about Power Spectral Density for power signals.

As we are in a stochastic context the autocorrelation is defined as : $$R_{xx}(\tau) = E[x(t)\overline{x(t-\tau)}]$$

Now if we consider $x$ to be a wide-sense stationary signal (i.e the mean and the covariance of the signal are constant with respect to time) the Wiener-Khinchin theorem (a nice proof here) tells us that :

$$PSD(\omega) = \int_{-\infty}^{+\infty} R_{xx}{\tau}e^{-i\omega \tau}d\tau$$

I am pretty sure there is a version of the Wiener-Khinchin theorem for discrete power signals.

It is even possible to go further than wide-sense stationary signals and use quasi-stationary signals. However in that context I don't know if there is an extension of the Wiener-Khinchin theorem, everytime I read about it the spectrum was defined as the Fourier transform of the autocorrelation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.