# Wiener Khinchin theorem : struggle in the derivation

I am reading Introduction to quantum noise, measurement and amplification, and I need to understand the Wiener Khinchin theorem: how to derive it. I also need to understand some context around this theorem (why some object are defined the way they are).

The theorem is derived on the page 55 of this document. I will explain to you what I understand from the starting point of this theorem and where I am stuck.

I assume a stationary stochastic gaussian process. I call $$R_{XX}(\tau)=\langle X(\tau) X(0) \rangle$$ the autocorrelation function of a stochastic variable $$X$$. I assume the signals to be real.

We define the power spectral density of noise as:

$$S_{XX}(\omega) \equiv \lim_{T \to +\infty} \langle |X_T(\omega)|^2 \rangle$$

Where $$X_T(\omega)$$ is the windowed Fourier transforme of $$X$$ on a period $$T$$. Basically we define the signal $$X_T(t)$$ that is equal to $$X(t)$$ for $$t \in [-T/2,T/2]$$ and that vanishes elsewhere. The Fourier transform of this signal can thus be defined as:

$$X_T(\omega) = \frac{1}{\sqrt{T}} \int_{-T/2}^{+T/2} dt X_T(t) e^{i \omega t}$$ To be consistant we must define the inverse Fourier transform as:

$$X_T(t) = \frac{\sqrt{T}}{2 \pi} \int_{-\infty}^{+\infty} d \omega X_T(\omega) e^{-i \omega t}$$

First questions:

• Why do we take such "uncommon" definition for the Fourier transform (usually it is $$\frac{1}{\sqrt{2 \pi}}$$ for both sides (direct+reciprocal), or $$\frac{1}{2 \pi}$$ in front of one of the sides. Is there a theoretical motivation for this in this context ?
• Why do we need to window the function ? Is it to avoid technical difficulties with divergent integrals ?

Now, I go to the derivation and express where I am stuck. The goal is to show that $$R_{XX}(\omega)$$, the Fourier transform of the autocorrelation function corresponds to the noise power spectral density $$S_{XX}(\omega)$$. I define $$S^T_{XX}(\omega)$$ the quantity before the limit of $$T \rightarrow +\infty$$. I know how we can derive computing the inverse Fourier transform of $$S_{XX}(\omega)$$ directly as suggested in a link in the comment. However I am stuck on deriving it from the other side (I have $$S_{XX}(\omega)$$ on the left hand side and I make appear $$T.F(R_{XX})$$ on the right handside). It might look as a boring question but in the document I follow they always follow this way of proving equalities, and I really don't get the manipulation they do. To understand how they do I need to understand this "other way" of deriving the theorem.

$$S^T_{XX}(\omega)=\frac{1}{T} \int_{-T/2}^{+T/2} dt \int_{-T/2}^{+T/2} dt' \langle X_T(t) X_T(t') \rangle e^{i \omega (t-t')}$$

Now, I use the fact my process is stationnary. And I recognize the autocorrelation function:

I start from the definition (before taking the limit): $$S^T_{XX}(\omega)=\frac{1}{T} \int_{-T/2}^{+T/2} dt \int_{-T/2}^{+T/2} dt' R_{X_T X_T}(t-t') e^{i \omega (t-t')}$$

I do the change of variables: $$(t,t') \rightarrow (\tau=t, \tau'=t-t')$$. It gives me:

$$S^T_{XX}(\omega)=\frac{1}{T} \int_{-T/2}^{+T/2} d \tau \int_{t-T/2}^{t+T/2} d \tau' R_{X_T X_T}(\tau) e^{i \omega \tau}$$

At this point, I have absolutely no idea how we could proceed.

We can also try another method (which I think is the way they do in the document but I really don't understand their justification). We can do the change of variable $$(t,t') \rightarrow (v=t+t',u=t-t')$$. The Jacobian gives $$1/2$$ and the boundaries are the one I write below:

$$S^T_{XX}(\omega)=\frac{1}{T} \int_{-T}^{T} dv \int_{-B(v)}^{+B(v)} \frac{du}{2} R_{X_T X_T}(u) e^{i \omega u}$$

$$B(v)=v+T$$ for $$-T \leq v \leq 0$$ and $$B(v)=t-v$$ for $$0 \leq v \leq T$$. It gives:

$$S^T_{XX}(\omega)=\frac{1}{T} \int_{0}^{T} dv \int_{-(T-v)}^{+(T-v)} \frac{du}{2} R_{X_T X_T}(u) e^{i \omega u}+\frac{1}{T} \int_{-T}^{0} dv \int_{-(T+v)}^{+(T+v)} \frac{du}{2} R_{X_T X_T}(u) e^{i \omega u}$$

And same problem here: I really don't see how we could get to the result with this way of deriving (which I think is roughly the path taken is the document I linked)

Thus, second question: How can prove the theorem with this method ?

• Is this answer what you're looking for? May 27 '20 at 15:08
• @Matt L. : Setting the integral of a complex exponential equal to a Dirac delta is a very useful heuristic, but it is not rigorous. The integral of a complex exponential over the whole real line does not converge. May 27 '20 at 19:17
• @JoeMac: It's a bit more than a useful heuristic, it's actually correct if interpreted correctly. It's an identity for distributions. May 27 '20 at 20:20
• @Matt L : Yes, there are ways to interpret a Dirac delta completely rigorously, but they require multiplying the complex exponential by a very-well-behaved function and then integrating. And we don't know if the functions here are Schwartz functions or infinitely differentiable functions of compact support. The rigorous yet concrete approach here is to integrate the complex exponential from -R to R, use properties of the sinc function, and then, at the end, let R grow to infinity. May 27 '20 at 20:33
• @MattL. thank you for your comment. Your answer was instructive but I would like to understand what is wrong with my attempt to derive it "from the other way". Indeed I need to understand this way of proceeding as in the document I attached they often do similar derivations. As I need to understand this paper I need to understand this other method. Thank you very much May 27 '20 at 21:24

Let me state at the beginning that the details that make this rigorous do not bring any extra understanding of the statistical behavior of signals, so the desire to use the Dirac delta distribution is completely understandable. If I could have figured out how to use it here rigorously, I would have.

I am going to repeatedly skip proof of the validity of "bringing expectation inside the integral". Note that computing an expectation is yet another integral (and over a finite measure space), so it really is just another change in the order of integration. To do so rigorously, appeal to Tonelli's Theorem and then Fubini's Theorem.

Let $$x$$ be a continuous-time wide-sense stationary stochastic process. $$\begin{eqnarray} r_{xx}(\tau) &=& \mathsf{E}\left[x(t)x(t+\tau)\right]\\ X_T(\omega) &=& \frac{1}{\sqrt{T}}\int_{-T/2}^{T/2}x(t)e^{-i\omega t}dt \end{eqnarray}$$ We assume that the integral defining $$X_T(\omega)$$ converges for almost all realizations of the process $$x$$. $$\begin{equation} \begin{split} \mathsf{E}\left[X_{T}(\omega)\overline{X_{T}(\omega)}\right] &=~ \mathsf{E}\left[\frac{1}{\sqrt{T}}\int_{-T/2}^{T/2}x(t)e^{-i\omega t}dt\overline{\frac{1}{\sqrt{T}}\int_{-T/2}^{T/2}x(t')e^{-i\omega t'}dt'}\right]\\ &=~ \mathsf{E}\left[\frac{1}{T}\int_{-T/2}^{T/2}\int_{-T/2}^{T/2}x(t)x(t')e^{-i\omega(t-t')}dtdt'\right]\\ &=~ \frac{1}{T}\int_{-T/2}^{T/2}\int_{-T/2}^{T/2}\mathsf{E}\left[x(t)x(t')\right]e^{-i\omega(t-t')}dtdt'\\ &=~ \frac{1}{T}\int_{-T/2}^{T/2}\int_{-T/2}^{T/2}r_{xx}(t-t')e^{-i\omega(t-t')}dtdt'\\ &=~ \frac{1}{T}\int_{-T/2}^{T/2}\int_{-T/2}^{T/2}\color{red}{r_{xx}(t-t')e^{-i\omega(t-t')}}dtdt' \end{split} \end{equation}$$

Note the presence of $$t-t'$$ as the argument of factors in the integrand. Our final goal is to have the Fourier transform of $$r_{xx}$$ on the right-hand side. Perhaps we can convert this to an iterated integral in $$t-t'$$ and a different variable. Let $$\begin{eqnarray} p &=& t - t',\\ q &=& t + t'. \end{eqnarray}$$ We want to change this to an iterated integral over $$p$$ and $$q$$. We are parametrizing the $$(t,t')$$ points with $$p$$ and $$q$$, and the Jacobian determinant of the map from $$(p,q)$$ to $$(t,t')$$ is $$\frac{1}{2}$$, so $$dt~dt' = \frac{1}{2}dp~dq$$.

The messy details that remain are describing the region of integration in terms of $$p$$ and $$q$$. The set over which we integrate is the diamond with corners at $$(0,T)$$, $$(T,0)$$, $$(0,-T)$$, and $$(-T,0)$$ in the $$(p,q)$$-plane. For $$-T\leq p \leq 0$$, we integrate $$dq$$ from $$q = -T-p$$ up to $$q = T+p$$. Hence the integral over $$-T \leq p \leq 0$$ is $$\begin{equation} \begin{split} &~\frac{1}{T}\int_{-T}^{0}\left[\int_{-T-p}^{T+p}dq\right]r_{xx}(p)e^{-i\omega p}\frac{1}{2}dp\\ ~=&~\frac{1}{T}\int_{-T}^{0}\left[T+p-(-T-p)\right]r_{xx}(p)e^{-i\omega p}\frac{1}{2}dp\\ ~=&~ \frac{1}{T}\int_{-T}^{0}\left(2T + 2p\right)r_{xx}(p)e^{-i\omega p}\frac{1}{2}dp\\ ~=&~ \int_{-T}^{0}\left(1 + \frac{p}{T}\right)r_{xx}(p)e^{-i\omega p}dp \end{split} \end{equation}$$

Now I add a restriction that a more seasoned professional might not need: I assume that $$\begin{equation} \int_{-\infty}^{\infty}\left|p~r_{xx}(p)\right|dp < \infty. \end{equation}$$ Then $$\begin{equation} \begin{split} &~\lim_{T\to\infty}\int_{-T}^{0}\left(1 + \frac{p}{T}\right)r_{xx}(p)e^{-i\omega p}dp\\ &=~\lim_{T\to\infty}\int_{-T}^{0}r_{xx}(p)e^{-i\omega p}dp + \lim_{T\to\infty}\frac{1}{T}\underbrace{\color{red}{\int_{-T}^{0}p~r_{xx}(p)e^{-i\omega p}dp}}_{\color{red}{\textrm{bounded as T\to\infty}}}\\ &=~ \int_{-\infty}^{0}r_{xx}(p)e^{-i\omega p}dp \end{split} \end{equation}$$

A similar calculation for $$0\leq p \leq T$$ (for which $$-T+p\leq q\leq T-p$$) yields $$\begin{equation} \int_{0}^{\infty}r_{xx}(p)e^{-i\omega p}dp. \end{equation}$$ Adding these shows that $$\begin{equation} \lim_{T\to\infty}\mathsf{E}\left[\left|X_T(\omega)\right|^2\right] ~=~ \int_{-\infty}^{\infty}r_{xx}(p)e^{-i\omega p}dp. \end{equation}$$

• Thank you for your answer. About the 1/2 it comes from the fact that as the integrals are between $[-T/2,+T/2]$ then the change of variable brings integral in $[-T,T]$. But this is not my issue here. I am not sure to understand your first equation right after "Hence the integral over −2T≤p≤0 is". I would expect to have: $\int_{-T}^{0} dp \int_{-(T+p)}^{T+p} \frac{du}{2} r_{xx}(u) e^{i \omega u}$. (I corrected for the $T \rightarrow T/2$ here). How do you make appear the $(2T+p)-(-2T-p)$ ? As the inner integral bounds depends on $p$, it is not as if we integrated a constant over $p$ May 28 '20 at 8:27
• Also, about your extra hypothesis of bounded integral. I am not supposed to need here right ? Do you mean that a very rigorous derivation of those quantities would not require this hypothesis and the conclusion would still hold ? Could you give a few words about it ? Thanks ! May 28 '20 at 8:34
• The argument of $r_{xx}$ and of the complex exponential is $p$, not $u$. We integrate $du$ ($dq$ in my notation) from $-T-p$ to $T+p$. Nothing else depends on $u$, so the integral in $u$ yields $2T+2p$. That is all "inside" the integral over $p$. May 28 '20 at 12:15
• I cannot provide a reference for my suspicion, but my impression after skimming references is that assuming integrability of $|p r_{xx}(p)|$ may be a stronger restriction than is truly needed. But I did not find an argument that helps me avoid it. Note that we would be alright if its integral from $-T$ to $T$ were bounded by $T^{1-\epsilon}$ for some positive $\epsilon$. May 28 '20 at 12:29
• Allright I agree with your calculations. In my answer my inner integral was on $u$ and not on $p$ which made the thing more complicated. I still have to redo all the thing from the beginning to check that everything is fine, I will do it later and come back if I still have issues. Thanks ! May 28 '20 at 12:54