# Why is sampling not “idempotent”?

This answer describes succinctly what sampling a continuous signal means in the frequency domain:

\begin{align*} x(t)\sum_{n=-\infty}^{\infty}\delta(t-nT) &\leftrightarrow \frac{1}{T}\sum_{k=-\infty}^{\infty} X\left(f-\tfrac{k}{T}\right) = \hat{X}(f) \end{align*}

Intuitively, this seems clear. “Sampling” actually means multiplying by a Dirac comb where the distance between the pulses is the sampling period, and because multiplication in one domain becomes convolution in the other, the Fourier transform of the Dirac comb, which is also a Dirac comb, convolves the spectrum of the original signal into infinitely many copies of that spectrum (which may overlap and combine if the signal was not band limited properly).

What I don’t understand, and I’m sure I’m either just missing something very basic, or making a simple mistake, is why this operation is not “idempotent”. That is, if I sample the resulting signals again, with the same parameters (i.e. identical sample rate and no offset), I would expect the spectrum to stay the same, because I would except the signal to not actually change.

I am sampling at exactly the same points I’ve sampled before. No support in the time domain should disappear, and none should be added.

Yet, if I look at the frequency domain, I’m adding together copies of the spectrum again. But unlike before, the spectrum already consists of infinitely many copies of the spectrum, so now I’m copying those again, and adding them altogether, which should yield infinity at every point that the spectrum was not zero? What am I missing?

• Sampling a dirac delta function is kinda hard. Infinitely hard. You have to hit something that has arbitrarily small width. The math guys tell us that dirac delta functions aren't really functions, but are "distributions". But we engineers do treat them like a function that is a limit of another well-defined function. So you can ask an approximation to the question with "approximations" to the dirac delta that has finite width. But it won't work when you let the width go to zero. Commented May 25 at 22:18
• Try making use of this Fourier series: $$\sum_{n=-\infty}^{\infty}\delta(t-nT) = \sum_{k=-\infty}^{\infty} \tfrac{1}{T} e^{i 2 \pi (k/T) t }$$ and see what you get. Commented May 25 at 22:22

## 4 Answers

Sampling” actually means multiplying by a Dirac comb

No. Sampling means turning a continuous function into a discrete set of numbers, so the process is simply

$$x[n] = x(nT_\mathrm{s})$$

where $$T_\mathrm{s} = \frac{1}{f_\mathrm{s}}$$ is the sampling period and $$f_\mathrm{s}$$ is the sample rate.

If you are interested in the spectrum of the discrete signal you will need to use a Fourier Transform flavor that can take in a set of numbers. These flavors are the DTFT and the DTF.

The whole math with the Dirac comb comes into place during reconstruction, i.e. when you try to turn the set of numbers back into a continuous signal. That requires you to assume what the digital to analog converter is doing and assuming a series of weighted Dirac's is a mathematically convenient (if supremely impractical) assumption. So you end up with

$$x_D(t) = \sum_n x(nT_\mathrm{s}) \delta(t-nT_\mathrm{s})$$

Note that the weighted Dirac signal is NOT the same as the original signal, i.e. $$x_D(t) \neq x(t)$$. For starters $$x_D(t)$$ has infinite bandwidth, so trying to sample it directly would be pointless. You can't sample a Dirac. However, you can calculate the spectrum of $$x_D(t)$$ and show that if you apply a suitable lowpass filter, you end up with the original spectrum and hence the original signal. After low passing the weighted Dirac comb, you can certainly sample again and get the same result.

• Genuine question. Are you saying that $x[n] = x(t-nT_{s}) \neq \int_{n}x(t)\delta(t-nT_{s})dt$? Not trying to misrepresent your point, so if I am please correct me, but that's what the first part of your response seems to imply. Commented May 26 at 15:31
• What I'm saying here is that the expression on the left of your first equation is not a discrete signal but a continuous one. It's well defined at all times $t$ and you use the continuous Fourier Transform to calculate the spectrum. Commented May 26 at 18:38
• The first equation in this answer should be $x[n]=x(nT_s)$. The way it is written now doesn't make sense because $x(t-nT_s)$ is a continuous-time function, not a sample value. Commented May 26 at 18:59
• //"The whole math with the Dirac comb comes into place during reconstruction, i.e. when you try to turn the set of numbers back into a continuous signal."// - - - - - I agree that this math is used to derive the reconstruction formula, but it's not solely there. Because $$x(t) \sum_{n=-\infty}^{\infty} \delta(t-nT) = x(t) \sum_{k=-\infty}^{\infty} \tfrac1T e^{i 2 \pi (k/T) t}$$ that math can be used to show the spectrum of $x(t)$ is periodically repeated and overlapped and added. Then from that, we can conclude that there better not be any frequency components in $x(t)$ at or above Nyquist. Commented May 26 at 19:27
• @MattL. I can't speak for Hilmar, but I think he is committed to science and correct mathematics. Just edit the answer to correct it. You gotta enuf rep. Commented May 26 at 19:29

Here is what I think is a nice pictoral way to think through this question.

Sadly there is signal processing literature out there that does equate multiplication by a Dirac comb with sampling. But this is incorrect. The correct statement about multiplication by the Dirac comb is that it gives us a continuous model of the effects of sampling, and in particular we want to argue interactions between band-limitedness and sampling and adverse effects such as aliasing.

The picture of overlapping frequency bands of signals that exceed a bandlimit is widely used, and being able to draw these pictures in my mind is the main source of this particular model of sampling.

The best way to explain this was given in the DSP context by Ken Steiglitz (it was his dissertation work of 1965!) but is available in accessible form in his textbook "A Digital Signal Processing Primer" (1996). This figure is my reproduction of the one in his text:

Steiglitz's Six Domains of Signal Processing, A DSP Primer (1996) p.220

The key is that sampling changes the domain. Sampling is taking a discrete subset of the real line. It turns out that this change in domain forces a change in the frequency domain as well. This is well known. By inspection we see that the Fourier transform of a sampled signal is bounded in a $$2\pi$$ long interval.

To explain in what sense the Dirac comb and its continuous Fourier Transform is a model of sampling start at the second row of Steiglitz's Six domains. When will all signals (time and frequency) in the first row form isomorphisms? (i.e. we are considering the rectangle of maps of the first and second row). The key is the Fourier transform. We "guess" that the Dirac comb is correct for the reason that the Fourier transform integral will not pick up any zeros (the "guess" here is mostly to dodge the need to discuss distributions, we do get all the correct intuition though). Hence nothing is added to the Fourier transforms for the flat zero sections between the Dirac deltas. So we can expect that the Fourier sum over discrete samples and the Fourier integral over the continuous Dirac comb will agree!

The full explanation of the right hand side can be found in Steiglitz's thesis but requires some mastery of working with Hilbert space methods. There is an underlying reason is topological and goes back to Pontryagin. There are a bunch of dualities as a consequence of what is known as Pontryagin duality. The one we need is the duality between discrete domains and "compact" (for our purposes think circular) domains. The Fourier dual of one always has to be the other! Some fairly introductory exposition of this can be found in lecture notes of a tutorial I presented at DAFx. But we don't need to understand this in detail when we are willing to believe Steiglitz's figure. Aliasing is the relationship of the Fourier dual of the unsampled domain to the dual of the sampled domain. I.e. it is the consequence of turning an infinite spectrum defined on $$\mathbb{R}$$ into a "compact" spectrum defined on the flat circle $$\mathbb{R}/\mathbb{Z}=\mathbb{S}^1$$. Think of a piece of $$\mathbb{R}$$ of some length, let's say $$2\pi$$ being wound around the circle. If we now roll that circle along an infinite line $$\mathbb{R}$$ how does it have to look to be the same. The answer is of course that any function of length $$2\pi$$ must be repeated on $$\mathbb{R}$$. This is precisely what the Fourier transform of the Dirac comb describes. Any convolution of a bandlimited function in $$2\pi$$ with the Dirac comb is precisely a function that can live on a $$2\pi$$ circle circumference. I.e. the Dirac comb in the frequency domain captures the repetition of going around the circle which carries the one circular frequency spectrum, over and over again.

And in this sense the Dirac comb and its Fourier dual are continuous models of sampling. It's precisely the isomorphisms around the Fourier transforms you get when you sample a Nyquist-bandlimited signal keeping track of all topological changes in the domains.

It's a worthwhile exercise to think through what this means for a signal that is sampled but is not bandlimited to Nyquist. Can one get an isomorphism following the above argument? If no, perhaps this is a neat way to think about perfect reconstruction and loss of information... Furthermore, if you know your six domains of signal processing and the discrete-to-compact duality, you will always know when your convolutions are circular!

The problem here is the mathematical model you're using. In practice there is no such thing as a Dirac comb. For some mathematical manipulations, the model of multiplying a continuous-time function with a Dirac comb is useful, but there are instances where the model fails to give reasonable results, and you've found one such case.

Mathematically speaking, the problem is that you can't sample a (weighted) Dirac comb, because that would mean evaluating a Dirac impulse, which can't be done because a Dirac impulse is not an ordinary function. The square of a Dirac impulse - which you'd get if you were to sample a sampled signal according to the model - is undefined.

If you replaced the Dirac impulses with rectangular pulses of unity height and width equal to the sampling interval, sampling a sampled signal would be well-defined, and repeated sampling wouldn't change anything.

Also take a look at this and this question and their answers.

While viewing sampling from the perspective of a dirac comb is correct and can be helpful, I find it is not the most complete or rigorous description of sampling. Sampling is best viewed, in my opinion, as an orthobasis expansion, analogous to the Fourier series (the Fourier series says a time limited signal can be mapped to a discrete set of frequencies, sampling theorem says a bandlimited signal can be reconstructed from a discrete sequence of time domain samples).

Basis representations analyze a signal as a linear combination, or superposition, of coefficients and basis functions. This is represented as

$$$$x(t) = \sum_{\gamma \in \Gamma} \underline{\alpha}(\gamma) \underline{\psi}_{\gamma}(t)$$$$

where $$$$\begin{bmatrix} \langle x(t),\psi_{1}(t) \rangle \\ \langle x(t),\psi_{2}(t) \rangle \\ \vdots \\ \langle x(t),\psi_{\gamma}(t) \rangle \\ \end{bmatrix} = \begin{bmatrix} \alpha_{1} \\ \alpha_{2} \\ \vdots \\ \alpha_{\gamma} \end{bmatrix} = \underline{\alpha}$$$$ Here, $$\langle \cdot \rangle$$ denotes an inner product.

With the Fourier series, the basis vectors are $$\psi_{k}(t) = e^{j2\pi kt}$$ and $$$$\underline{\alpha} = \begin{bmatrix} \langle x(t),e^{j2\pi k_{1}t} \rangle \\ \langle x(t),e^{j2\pi k_{2}t} \rangle \\ \vdots \\ \langle x(t),e^{j2\pi k_{\gamma}t} \rangle \\ \end{bmatrix}$$$$

Sampling theorem says that we can reconstruct sufficiently bandlimited $$x(t)$$ from point samples equally spaced by $$T$$ $$$$x[n] \triangleq x(nT)$$$$ $$$$x(t) = \sum_{n=-\infty}^{\infty}x[n] \frac{\sin\left(\frac{\pi(t-nT)}{T}\right)}{\frac{\pi(t-nT)}{T}}$$$$

We can rewrite this using basis expansion principles by saying $$$$x(t) = \sum_{n=-\infty}^{\infty}\alpha(n)\psi_{n}(t)$$$$ $$$$\psi_{n}(t) = \frac{\sin\left(\frac{\pi(t-nT)}{T}\right)}{\frac{\pi(t-nT)}{T}}$$$$ $$$$\alpha(n) = x(nT)$$$$

We know that $$x[n] = x(t) \: \forall \: t = nT$$. So, if we want to see the effect of sampling $$x[n]$$, we plug in $$k\tau$$ for $$nT$$ to represent the new sampling parameters, and then set $$t = nT$$ to represent sampling $$x[n]$$. If $$k\tau = nT$$, then you get an exact reconstruction of $$x[n]$$. If $$\tau \neq T$$ (different sample rates) or $$k\tau = nT + \epsilon$$ (arbitrary shift), you will not be guaranteed a reconstruction of $$x[n]$$.

EDIT: For discussion purposes

Per a discussion in the comments. I am adding thoughts here to prevent an overflow in the comment section while still being able to express mathematical thought.

According to the posts here and here, and in alignment with what I know the Shannon-Whittaker interpolation formula is, we can write the reconstruction as \begin{aligned} x(t) &= \sum_{n=-\infty}^{\infty}x[n]\operatorname{sinc}(\frac{t-nT}{T})\\ &= \sum_{n=-\infty}^{\infty}x[n]\operatorname{sinc}(\frac{t}{T}-n)\\ &= \sum_{n=-\infty}^{\infty}x[n]\frac{\sin(\pi(\frac{t-nT}{T}))}{\pi(\frac{t-nT}{T})}\\ &= \sum_{n=-\infty}^{\infty}x[n]\frac{T\sin(\pi(\frac{t-nT}{T}))}{\pi(t-nT)} \end{aligned} I'm not understanding how, if this is wrong, where I am going wrong.

As for the other two comments, for $$\phi_{k}(t)$$ for the Fourier series, I used subscripts to keep the notation as general as possible, but do realize how that could be interpreted as $$k \notin \mathbb{Z}$$. Additionally, I'm not so sure that the vector $$\alpha$$ couldn't be countably infinite length. My most recent notes on this though were in the context of sparsity though, hence I just copied the notation, so I'd have to go back and reconsider that.

While many of these points may be technically valid, I don't see how these objections significantly alter the implications of my post, ie that sampling is an orthobasis expansion and this notation is far easier to interpret the problem at hand through than Dirac comb notation, IMO.

• You might have a few mistakes. I corrected one that was obvious. Still looking at this carefully. Commented May 26 at 20:09
• //"... the Fourier series says a time limited signal can be mapped to a discrete set of frequencies,"// - - - - - while true, it's even more true to say that "...the Fourier series says a time limited signal can be periodically extended and mapped to a discrete set of frequencies," When "time-limited", there is strictly no definition of the signal outside of its domain. But that doesn't stop some people from assuming it's zero outside of the domain. For Fourier series the better assumption is that it's periodically extended, not zero extended. Commented May 26 at 20:13
• @robertbristow-johnson to the first comment, I made an assumption that, since $x(t)$ is continuous, it is band-limited to $[-\pi/T,\pi/T]$. See en.m.wikipedia.org/wiki/… for the relationship Commented May 26 at 20:24
• @robertbristow-johnson please see my edit Commented May 27 at 1:45
• You're right, we're both wrong. I didn't really correct your equation, which originally was not exactly what you got in your edit. Commented May 27 at 4:04