Sampling Theorem and Signals Explained to a Mathematician

Question

Let $f:\left(-\frac T2,\frac T2\right)\to\mathbb{R}$ for some $T>0$. The Fourier coefficients of $f$ are $$\left\{\begin{matrix}a_0&=&\displaystyle\frac 1T\int_{-T/2}^{T/2}g(t)\;dt\\a_k&=&\displaystyle\frac 2T\int_{-T/2}^{T/2}g(t)\cos\left(k\frac{2\pi}Tt\right)\;dt\\b_k&=&\displaystyle\frac 2T\int_{-T/2}^{T/2}g(t)\sin\left(k\frac{2\pi}Tt\right)\;dt\end{matrix}\right.\;\;\;\;\;(k\in\mathbb{N})$$ and the Fourier polynom of degree $n\in\mathbb{N}$ itself is $$\mathcal{F}_n[f](x):=a_0+\sum_{k=1}^n\left(a_k\cos\left(k\frac{2\pi}Tx\right)+b_k\sin\left(k\frac{2\pi}Tx\right)\right)\tag{1}$$ Now, for any $g\in L^2$, where $$L^2:=\left\{g:(-\pi,\pi)\to\mathbb{C}:g\text{ is continuous almost everywhere}\right\},$$ the complex Fourier coefficients $c_k$ of $g$ satisfy Parseval's identity, i.e. $$\sum_{k\in\mathbb{Z}}\left|c_k\right|^2=\frac{1}{2\pi}\left\|g\right\|_{L^2}^2\tag{2}$$

Now, we can interpret $f$ as a $T$-periodic signal that we want to transmit over a channel. I'm asking myself the following questions:

1. What is the signal bandwidth?$^1$
2. What is the frequency spectrum?
3. What is the channel bandwidth?
4. What does the sampling theorem state?
5. I've read that the square of the $k$-th coefficient, i.e. $a_k^2+b_k^2$, is proportional to the "energy contained in this harmonic". I think this is somehow related to $(2)$. But I have no idea what is meant by energy.

I've read all the definitions, but (as I'm a computer science and mathematics student with almost no relation to physics) I wasn't able to answer the questions above by myself.

I know that $(4.)$ has something to do with the fact, that we need to use some kind of discrete Fourier transformation, i.e. we consider an equidistant grid $$x_j=-\pi+j\frac \pi N\;\;\;\text{for }j=0,\ldots,2N$$ and (maybe) use the composite trapezoidal rule to approximate $$c_k=\frac{1}{2\pi}\int_{-\pi}^\pi f(x)e^{-ikx}\;dx\approx\frac{1}{2N}\sum_{j=0}^{2N-1}f\left(x_j\right)\exp\left(-ik\frac{2\pi}Tx_j\right)$$ So, I think the sampling theorem targets the question how huge do I need to choose $N$ in order to reconstruct (what exactly does that mean? If the Fourier polynom has infinitely many summands we cannot reconstruct the original function) the signal and at which size of $N$ it would be pointless to choose an even bigger $N$.

---

$^1\;\;$My lecture notes distinguish whether or not $\displaystyle\mathcal{F}[f]:=\lim_{n\to\infty}\mathcal{F}_n[f]$ is actually finite (i.e. $\exists k_0\in\mathbb{N}:\forall k\ge k_0:a_k=b_k=0$). They state, that "the signal bandwith is the difference between the lowest and the highest frequency"? What does that mean? In the first place, I wasn't even sure if signal bandwidth is a property of $f$ or $\mathcal{F}[f]$. Considering $f$, I would not understand why $f$ should have multiple frequencies and how I would calculate them. So, I think we need to consider $\mathcal{F}[f]$. More precisely, I think we need to look at the "frequencies" of the sine and cosine functions in the $k$-th summand in $(1)$. Obviously, these frequencies are both equal to $$\mathcal{f}_k:=\frac kT$$ and independent of $f$. But, I still don't understand how I can determine the "highest" frequency.

Answer 1

"Highest frequency" is shorthand for the highest k/T for which a and b sub k are non-zero.

Considering f, one reason it might consist of the sum of multiple sinusoidal components is that this is a solution to many ordinary linear low-order differential equations.

Answer 2

It seems like you have a lot of fundamental questions. The following blog might be useful for you:

http://python-for-signal-processing.blogspot.com/2012/09/investigating-sampling-theorem-in-this.html

The material there is also downloadable as an IPython Notebook, which means you can do the computations yourself in Python to play with the concepts that are discussed. There is also a corresponding text by Springer by the same author which collects most of the material. on the blog.

Answer 3

Note that I'll use 'function of time' and 'signal' interchangeably below. I'm an engineer, so please excuse my informal notation and definitions. I won't try to provide exhaustive answers, only pointers. At the end I give a link to a book that should prove very useful to you.

1. There are many definitions of bandwidth. One example is: the smallest positive frequency beyond which all Fourier coefficients are zero. Another example is: the smallest positive frequency $f_0$ such that 99% of the signal's power is contained between $f=0$ and $f=f_0$.

   Generally, you choose the definition of bandwidth that is more appropriate to the application you're studying.

2. A signal's spectrum is its Fourier transform or series. In the case of the Fourier series, it is a one-to-one mapping from frequency to $(a_k,b_k)$ tuples. It is common to use 'spectrum' to refer to a plot of the Fourier transform or series.

3. You can think of a channel as a filter; that is, a system that outputs $H(f_0)\cos(2\pi f_0t)$ when the input is $\cos(2\pi f_0t)$, for a given complex function $H(f)$. The channel's bandwidth is the range of frequencies $f$ for which $H(f)$ is non-zero (or sufficiently large for your application). It may also mean the range of frequencies for which $H(f)$ is a non-zero constant.

4. Hard to answer in detail here, but the theorem gives sufficient conditions under which it is possible to take an analog signal, sample it, and rebuild the signal from its samples. The theorem assumes an infinite-duration signal of limited bandwidth and an ideal interpolator.

5. In practice we use analog voltages instead of mathematical functions. A voltage only exists across a resistance, which implies current flow through the resistor. In turn, this implies energy expenditure. So, a signal's energy is the amount of joules it takes to generate it as a voltage across a given resistor. This concept can be extended to frequency, that is: how much of the total energy is contributed by each individual harmonic.

It seems to me that you'll enjoy this book: A foundation in digital communication by A. Lapidoth. It's free, and it will answer your question in much more detail than is possible here.