Proving Nyquist Sampling Theorem for Strictly Band Limited Signals (Whittaker Shannon Interpolation Formula)

Question

I understand that the Nyquist sampling theorem dictates that the minimum sampling frequency, $f_s$, be s.t. $f_s > 2B$, where $ B $, is the bandwidth of the signal. I have read the explanation for what happens when the input signal contains an impulse/non zero ferquency spectrum at $f_s = f_m$. But is there any way to do a mathematical proof that sampling at $f_s = 2B$ works when the signal is strictly bandlimited.

A little background : This is a HW problem

The given hint was to use the fact that "one way to interpret Nyquist Sampling theorem is to note that any bandlimited signal can be represented as a superposition of bandlimited signals that are orthogonal to each other."

Answer 1

Approaching The Sampling Theorem as Inner Product Space

Preface

There are many ways to derive the Nyquist Shannon Sampling Theorem with the constraint on the sampling frequency being 2 times the Nyquist Frequency.
The classic derivation uses the summation of sampled series with Poisson Summation Formula.

Let's introduce different approach which is more similar to function analysis - Building an orthogonal space and using projection for analysis and synthesis.

Forming Orthonormal Basis

In this section we'll define an orthonormal base and derive the decomposition and composition process.

Definitions

First one should define the space of Band Limited Functions. The space of Band Limited Functions is defined by:

$$ \mathcal{B}_{ {W}_{s} } = \left\{ f \left( x \right) \mid F \left( \omega \right) = \mathscr{F} \left\{ f \left( x \right) \right\}, \; F \left( \omega \right) = 0, \; \forall \, \left| \omega \right| > {W}_{s} \right\} $$

In words, it means that for each function $ f \left( x \right) \in \mathcal{B}_{ {W}_{s} }$ its fourier transform $ F \left( \omega \right) = \mathscr{F} \left\{ f \left( x \right) \right\} $ vanishes for frequencies $ \left| \omega \right| > {W}_{s} $.

The inner product in this space is given by:

$$ \langle f \left( x \right), g \left( x \right) \rangle = \frac{1}{T} \int_{- \infty}^{\infty} f \left( x \right) g \left( x \right) dx, \; T = \frac{2 \pi}{ {W}_{s} } $$

One could easily show that this is a indeed an inner product space with valid inner product.

The main claim is the orthonormal basis of this space is given by:

$$ f \left( x \right) = \operatorname{sinc} \left( \frac{ x - n T }{T} \right) $$

Where $ \operatorname{sinc} \left( x \right) $ is the Normalized Sinc Function given by $ \operatorname{sinc} \left( x \right) = \frac{ \sin(\pi x) }{ \pi x } $.
The basis functions are parameterized by the parameter $ n $. Basically we have shifted and scaled function as the basis.

Proof of The Orthonormal Property

One must show the orthonormal property of the basis under the defined inner product:

$$ \begin{aligned} \langle f \left( x \right), g \left( x \right) \rangle & = \frac{1}{T} \int_{- \infty}^{\infty} f \left( x \right) g \left( x \right) dx = \frac{1}{T} \int_{- \infty}^{\infty} \operatorname{sinc} \left( \frac{ x - n T }{T} \right) \operatorname{sinc} \left( \frac{ x - m T }{T} \right) dx && \text{} \\ & \overset{1}{=} \frac{1}{T} \int_{- \infty}^{\infty} \left( \frac{1}{2 \pi} \int_{- \infty}^{\infty} T \, \Pi \left( \frac{ \omega }{ {W}_{s} } \right) {e}^{-j \omega n T} {e}^{j \omega t} d\omega \right) \operatorname{sinc} \left( \frac{ x - m T }{T} \right) dx && \text{} \\ & \overset{2}{=} \frac{1}{T} \int_{-\infty}^{\infty} \frac{1}{2 \pi} T \, \Pi \left( \frac{\omega}{ {W}_{s} } \right) {e}^{-j \omega n T} \left( \int_{- \infty}^{\infty} {e}^{j \omega x} \operatorname{sinc} \left( \frac{x - m T}{T} \right) dx \right) d\omega && \text{} \\ & \overset{3}{=} \frac{1}{T} \int_{-\infty}^{\infty} \frac{1}{2 \pi} T \, \Pi \left( \frac{\omega}{ {W}_{s} } \right) {e}^{-j \omega n T} T \, \Pi \left( \frac{-\omega}{ {W}_{s} } \right) {e}^{j \omega m T} d\omega && \text{} \\ & \overset{4}{=} \frac{T}{2 \pi} \int_{ - \frac{ {W}_{s} }{2} }^{ \frac{ {W}_{s} }{2} } {e}^{j \omega \left( m - n \right) T} d\omega && \text{} \\ & \overset{5}{=} \begin{cases} 1 & \text{ if } m = n \\ 0 & \text{ if } m \neq n \end{cases} \end{aligned} $$

Where:

1. Since $ \mathscr{F} \left\{ \operatorname{sinc} \left( \frac{ x - n T }{T} \right) \right\} = T \, \Pi \left( \frac{ \omega }{ {W}_{s} } \right) {e}^{-j \omega n T} $.
2. Changing order of integration for converging integrals.
3. Applying $ \mathscr{F} \left\{ \operatorname{sinc} \left( \frac{ x - m T }{T} \right) \right\} \left( - \omega\right) $.
4. Integration boundaries according to the Rect function (Multiplication).
5. Integration over a cycle or over a constant.

Since we proved the suggested base is indeed an orthonormal basis of the space the following holds:

$$ \forall f \left( x \right) \in \mathcal{B}_{{W}_{s}} , \; f \left( x \right) = \sum_{n = -\infty}^{n} \langle f \left( x \right), {g}_{n} \left( x \right) \rangle {g}_{n} \left( x \right) $$

Where $ {g}_{n} \left( x \right) = \operatorname{sinc} \left( \frac{ x - n T }{T} \right) $ and $ \langle f \left( x \right), {g}_{n} \left( x \right) \rangle $ is the projection of $ f \left( x \right) $ on $ {g}_{n} \left( x \right) $.

Projection Process

As written above, using the result of a projection of a function in the space onto the basis one could reconstruct it as:

$$ f \left( x \right) = \sum_{n = -\infty}^{n} \langle f \left( x \right), {g}_{n} \left( x \right) \rangle {g}_{n} \left( x \right) $$

The question is, what's is the projection of a general function in this space? Well, it turns out it can be shown in a sloed form way:

$$ \begin{aligned} \langle f \left( x \right), {g}_{n} \left( x \right) \rangle & = \frac{1}{T} \int_{- \infty}^{\infty} f \left( x \right) {g}_{n} \left( x \right) dx = \frac{1}{T} \int_{- \infty}^{\infty} f \left( x \right) \operatorname{sinc} \left( \frac{ x - n T }{T} \right) dx && \text{} \\ & \overset{1}{=} \frac{1}{T} \int_{- \infty}^{\infty} \left( \frac{1}{2 \pi} \int_{-\infty}^{\infty} F \left( \omega \right) {e}^{j \omega x} d\omega \right) \operatorname{sinc} \left( \frac{ x - n T }{T} \right) dx && \text{} \\ & \overset{2}{=} \frac{1}{2 \pi T} \int_{- \infty}^{\infty} \left( \int_{- \infty}^{\infty} \operatorname{sinc} \left( \frac{ x - n T }{T} \right) {e}^{j \omega x} dx \right) F \left( \omega \right) d\omega && \text{} \\ & \overset{3}{=} \frac{1}{2 \pi T} \int_{- \infty}^{\infty} T \, \Pi \left( \frac{-\omega}{ {W}_{s} } \right) {e}^{j \omega n T} {F} \left( \omega \right) d\omega && \text{} \\ & \overset{4}{=} \frac{1}{2 \pi} \int_{ - \frac{ {W}_{s} }{2} }^{ \frac{ {W}_{s} }{2} } F \left( \omega \right) {e}^{j \omega n T} d\omega && \text{} \\ & \overset{5}{=} f \left( n T \right) \end{aligned} $$

Where:

1. Since $ \mathscr{F} \left\{ f \left( x \right) \right\} = F \left( \omega \right) $.
2. Changing order of integration for converging integrals.
3. Applying $ \mathscr{F} \left\{ \operatorname{sinc} \left( \frac{ x - m T }{T} \right) \right\} \left( -\omega \right) $.
4. Integration boundaries according to the Rect function.
5. Applying Inverse Fourier Transform at $ x = n T $.

Wrapping it yields:

$$ f \left( x \right) = \sum_{n = -\infty}^{n} \langle f \left( x \right), {g}_{n} \left( x \right) \rangle {g}_{n} \left( x \right) = \sum_{n = - \infty}^{\infty} f \left( n T \right) \operatorname{sinc} \left( \frac{ x - n T }{T} \right) $$

Which is known as the Whittaker Shannon Interpolation Formula.

Conclusion

In the process above the analysis and synthesis of Band Limited Functions is shown using Orthonormal Basis. If one set $ T $ to be the Sampling Interval, usually denoted by $ {T}_{s} $ then the Sampling Frequency is given by $ {F}_{s} = \frac{1}{ {T}_{s} } $.
Now, since the function in frequency domain stretches in the range $ \left[ -\frac{{F}_{s}}{2}, \frac{{F}_{s}}{2} \right] $ we indeed have the known relation that the sampling frequency has to be twice the one sided support of the function in frequency.