# Derivation of Nyquist Frequency and Sampling Theorem [closed]

I have been looking through different sites and questions over the internet about Sampling theory, but couldn’t find the clear definition of how nyquist frequency condition is derived? It would be great if someone could direct me to the derivation of the condition that sampling frequency should >= to maximum frequency in the signal

• The basic Wikipedia article en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem is fairly well written and shows multiple ways to derive it. If you need further help please indicate which parts of the standard deviations are a problematic for you – Hilmar Oct 11 '19 at 0:54
• I added a derivation from a different point of view (2 sided of the same coin of course) in comparison to Wikipedia. – Royi Oct 13 '19 at 0:14

# Approaching The Sampling Theorem as Inner Product Space

## Preface

There are many ways to derive the Nyquist Shannon Sampling Theorem with the constraint on the sampling frequency being 2 times the Nyquist Frequency.
The classic derivation uses the summation of sampled series with Poisson SummationFormula.

Let's introduce different approach which is more similar to function analysis - Building an orthogonal space and using projection for analysis and synthesis.

## Forming Orthonormal Basis

In this section we'll define an orthonormal base and derive the decomposition and composition process.

### Definitions

First one should define the space of Band Limited Functions. The space of Band Limited Functions is defined by:

$$\mathcal{B}_{ {W}_{s} } = \left\{ f \left( x \right) \mid F \left( w \right) = \mathcal{F} \left\{ f \left( x \right) \right\}, \; F \left( w \right) = 0, \; \forall \, \left| w \right| > {W}_{s} \right\}$$

In words, it means that for each function $$f \left( x \right) \in \mathcal{B}_{ {W}_{s} }$$ its fourier transform $$F \left( w \right) = \mathcal{F} \left\{ f \left( x \right) \right\}$$ vanishes for frequencies $$\left| w \right| > {W}_{s}$$.

The inner product in this space is given by:

$$\langle f \left( x \right), g \left( x \right) \rangle = \frac{1}{T} \int_{- \infty}^{\infty} f \left( x \right) g \left( x \right) dx, \; T = \frac{2 \pi}{ {W}_{s} }$$

One could easily show that this is a indeed an inner product space with valid inner product.

The main claim is the orthonormal basis of this space is given by:

$$f \left( x \right) = \operatorname{sinc} \left( \frac{ x - n T }{T} \right)$$

Where $$\operatorname{sinc} \left( x \right)$$ is the Normalized Sinc Function given by $$\operatorname{sinc} \left( x \right) = \frac{ \pi x }{ x }$$.
The basis functions are parameterized by the parameter $$n$$. Basically we have shifted and scaled function as the basis.

### Proof of The Orthonormal Property

One must show the orthonormal property of the basis under the defined inner product:

\begin{aligned} \langle f \left( x \right), g \left( x \right) \rangle & = \frac{1}{T} \int_{- \infty}^{\infty} f \left( x \right) g \left( x \right) dx = \frac{1}{T} \int_{- \infty}^{\infty} \operatorname{sinc} \left( \frac{ x - n T }{T} \right) \operatorname{sinc} \left( \frac{ x - m T }{T} \right) dx && \text{} \\ & \overset{1}{=} \frac{1}{T} \int_{- \infty}^{\infty} \left( \frac{1}{2 \pi} \int_{- \infty}^{\infty} T \Pi \left( \frac{ w }{ {W}_{s} } \right) {e}^{-j w n T} {e}^{j w t} dw \right) \operatorname{sinc} \left( \frac{ x - m T }{T} \right) dx && \text{} \\ & \overset{2}{=} \frac{1}{T} \int_{-\infty}^{\infty} \frac{1}{2 \pi} T \Pi \left( \frac{w}{ {W}_{s} } \right) {e}^{-j w n T} \left( \int_{- \infty}^{\infty} {e}^{j w x} \operatorname{sinc} \left( \frac{x - m T}{T} \right) dx \right) dw && \text{} \\ & \overset{3}{=} \frac{1}{T} \int_{-\infty}^{\infty} \frac{1}{2 \pi} T \Pi \left( \frac{w}{ {W}_{s} } \right) {e}^{-j w n T} T \Pi \left( \frac{-w}{ {W}_{s} } \right) {e}^{j w m T} dw && \text{} \\ & \overset{4}{=} \frac{T}{2 \pi} \int_{ - \frac{ {W}_{s} }{2} }^{ \frac{ {W}_{s} }{2} } {e}^{j w \left( m - n \right) T} dw && \text{} \\ & \overset{5}{=} \begin{cases} 1 & \text{ if } m = n \\ 0 & \text{ if } m \neq n \end{cases} \end{aligned}

Where:

1. Since $$\mathcal{F} \left\{ \operatorname{sinc} \left( \frac{ x - n T }{T} \right) \right\} = T \Pi \left( \frac{ w }{ {W}_{s} } \right) {e}^{-j w n T}$$.
2. Changing order of integration for converging integrals.
3. Applying $$\mathcal{F} \left\{ \operatorname{sinc} \left( \frac{ x - m T }{T} \right) \right\} \left( - w \right)$$.
4. Integration boundaries according to the Rect function (Multiplication).
5. Integration over a cycle or over a constant.

Since we proved the suggested base is indeed an orthonormal basis of the space the following holds:

$$\forall f \left( x \right) \in \mathcal{B}_{{W}_{s}} , \; f \left( x \right) = \sum_{n = -\infty}^{n} \langle f \left( x \right), {g}_{n} \left( x \right) \rangle {g}_{n} \left( x \right)$$

Where $${g}_{n} \left( x \right) = \operatorname{sinc} \left( \frac{ x - n T }{T} \right)$$ and $$\langle f \left( x \right), {g}_{n} \left( x \right) \rangle$$ is the projection of $$f \left( x \right)$$ on $${g}_{n} \left( x \right)$$.

### Projection Process

As written above, using the result of a projection of a function in the space onto the basis one could reconstruct it as:

$$f \left( x \right) = \sum_{n = -\infty}^{n} \langle f \left( x \right), {g}_{n} \left( x \right) \rangle {g}_{n} \left( x \right)$$

The question is, what's is the projection of a general function in this space? Well, it turns out it can be shown in a sloed form way:

\begin{aligned} \langle f \left( x \right), {g}_{n} \left( x \right) \rangle & = \frac{1}{T} \int_{- \infty}^{\infty} f \left( x \right) {g}_{n} \left( x \right) dx = \frac{1}{T} \int_{- \infty}^{\infty} f \left( x \right) \operatorname{sinc} \left( \frac{ x - n T }{T} \right) dx && \text{} \\ & \overset{1}{=} \frac{1}{T} \int_{- \infty}^{\infty} \left( \frac{1}{2 \pi} \int_{-\infty}^{\infty} F \left( w \right) {e}^{j w x} dw \right) \operatorname{sinc} \left( \frac{ x - n T }{T} \right) dx && \text{} \\ & \overset{2}{=} \frac{1}{2 \pi T} \int_{- \infty}^{\infty} \left( \int_{- \infty}^{\infty} \operatorname{sinc} \left( \frac{ x - n T }{T} \right) {e}^{j w x} dx \right) F \left( w \right) dw && \text{} \\ & \overset{3}{=} \frac{1}{2 \pi T} \int_{- \infty}^{\infty} T \Pi \left( \frac{-w}{ {W}_{s} } \right) {e}^{j w n T} {F} \left( w \right) dw && \text{} \\ & \overset{4}{=} \frac{1}{2 \pi} \int_{ - \frac{ {W}_{s} }{2} }^{ \frac{ {W}_{s} }{2} } F \left( w \right) {e}^{j w n T} dw && \text{} \\ & \overset{5}{=} f \left( n T \right) \end{aligned}

Where:

1. Since $$\mathcal{F} \left\{ f \left( x \right) \right\} = F \left( w \right)$$.
2. Changing order of integration for converging integrals.
3. Applying $$\mathcal{F} \left\{ \operatorname{sinc} \left( \frac{ x - m T }{T} \right) \right\} \left( - w \right)$$.
4. Integration boundaries according to the Rect function.
5. Applying Inverse Fourier Transform at $$x = n T$$.

Wrapping it yields:

$$f \left( x \right) = \sum_{n = -\infty}^{n} \langle f \left( x \right), {g}_{n} \left( x \right) \rangle {g}_{n} \left( x \right) = \sum_{n = - \infty}^{\infty} f \left( n T \right) \operatorname{sinc} \left( \frac{ x - n T }{T} \right)$$

Which is known as the Whittaker Shannon Interpolation Formula.

## Conclusion

In the process above the analysis and synthesis of Band Limited Functions is shown using Orthonormal Basis. If one set $$T$$ to be the Sampling Interval, usually denoted by $${T}_{s}$$ then the Sampling Frequency is given by $${F}_{s} = \frac{1}{ {T}_{s} }$$.
Now, since the function in frequency domain stretches in the range $$\left[ -\frac{{F}_{s}}{2}, \frac{{F}_{s}}{2} \right]$$ we indeed have the known relation that the sampling frequency has to be twice the one sided support of the function in frequency.

It really boils down to aliasing. In continuous-time, if you have any two signals $$x_1(t) = \sin(2 \pi F_1 t)$$ and $$x_2(t) = \sin(2 \pi F_2 t)$$, then as long as $$F_1$$ and $$F_2$$ are distinct, the signals are, too.

But consider sampling at some time interval $$T_s$$, so that the sampled signals are $$x_1(k) = \sin(2 \pi F_1 T_s k)$$ and $$x_2(k) = \sin(2 \pi F_2 T_s k)$$. If you have a condition where $$2 \pi F_1 = 2 \pi F_2 \pm 2\pi n$$ for any integer $$n$$, then the signals are not clearly not distinguishable, by the simple trigonometric identity $$\sin \theta = \sin \theta + 2\pi$$. Do the math, and you find that the frequencies cannot be separated by more than the sampling rate to be distinguishable.

It's worse, though, because you also have the trigonometric identity $$\sin \theta = -\sin \theta + \pi$$. When you consider that the only thing that allows you to distinguish one signal from another is the frequency, that means that if $$F_2$$ is a mere $$\frac{1}{2 T_s}$$ away from $$F_1$$, then $$x_1(t)$$ cannot be distinguished from $$x_2(t)$$.

This phenomenon of two distinct signals appearing as one after sampling is called aliasing. It's pretty much the basis of the Shannon-Nyquist sampling theorem, and the Nyquist rate. Because if you're sampling at $$\frac{1}{T_s}$$ and trying to distinguish signals further apart than $$\frac{1}{2T_s}$$, you just can't.

Hence, the Nyquist rate.