Conclusions of sampling around Nyquist Rate

Question

I'm trying to understand some results of playing around with sampling around a signal's Nyquist sampling rate. For my example, I'm sampling a $B=5\mathrm{Hz}$ wave over a 1 second period.

1. In the first image, I use a very high sampling rate of 1000Hz to emulate the exact signal (at least for graphing), $\sin(2 \pi 5 t)$. This sampling is subsequently used in the next graphs as a reference.
2. In the next, I use a rate just below Nyquist's, 9Hz. The resulting wave looks like a sine wave, but doesn't represent the highs and lows of the wave. I'm still tracking with why hitting the Nyquist rate is important from this image.
3. Next, I get confused with Nyquist: I sample at exactly $2B=10\mathrm{Hz}$, the lower bound. This produces nearly zeros in the data:

   [ 0.00000000e+00  1.22464680e-16 -2.44929360e-16 -1.40896280e-15
   -4.89858720e-16  6.12323400e-16  2.81792560e-15 -2.69546092e-15
   -9.79717439e-16  1.10218212e-15]
   

   (I'm not even sure if these are floating point errors or not). Yes, there is some oscillation, but it is very minor. It seems like we need to further increase the sample rate... Is the rate of $2B$ exclusive?
4. In the fourth image, I sample above at 11Hz, which above the Nyquist rate. However, this sampling looks very much like image #2 flipped about the x axis. Again, there is some oscillation, but I don't see how we could reconstruct a 5 Hz wave from this, especially when it has nearly the same qualities as image #2.
5. Finally, I sample at $4B=10\mathrm{Hz}$ and at last, I can see the 5Hz wave I expect. Why did it take 4x to get here when the theory only states 2x?

I feel like the conclusions I've made above are incorrect (in probably a variety of ways). Would someone please help me explain where I'm wrong?

Jupyter Notebook available here: https://gist.github.com/t-mart/8a46e87938904c7f6a5a102ad6a2ef0e

Answer 1

Is the rate of 2B exclusive?

Yes. The sampling theorem states that the signal must be band limited to half the sample rate. That implies that the energy at the Nyquist frequency must be zero. In practice you need a healthy margin between the highest usable frequency and the Nyquist frequency. There is always some "transition band" that you need to get the energy from "useful" to "nothing"

I don't see how we could reconstruct a 5 Hz wave from this

You can using Shannon Whittaker interpolation. https://en.wikipedia.org/wiki/Whittaker%E2%80%93Shannon_interpolation_formula It doesn't matter how it looks to your eye, it just matters that all the information is there, which it is.

Why did it take 4x to get here when the theory only states 2x?

It doesn't take 4x. You need to use a quantitative mathematical criteria to determine whether a sampling is "good" or not. Visual inspection doesn't help here. The most straight forward would be: if you send this sequence into a D/A converter and compare the output with the original input, how different are the two signals? That's actually what the Shannon interpolation does: it mimics an DAC converter with an infinitely steep anti-aliasing filter.

Answer 2

Take your sine wave at $f = 5\mathrm{kHz}$; $x(t) = \sin 2 \pi f\, t$. If you sample it at $F_s$ then you get a result $y[k] = \sin \frac{2 \pi f}{F_s} k$*. Because of the trigonometric identity $\sin \theta = \sin (2\pi + \theta)$, you can't tell the difference between sine waves at $f$ and $f \pm nF_s$. Because of the trigonometric identity $\sin \theta = -\sin (\pi + \theta)$, unless you know the starting phase (which you never do) you can't tell the difference between sine waves at $f$ and $f = \pm \frac{n}{2}F_s$.

This not being able to tell is where aliasing comes from.

Moreover, when you reconstruct a signal from the sampled-time domain into continuous-time, those ambiguities in the sine wave appear as actual signals in the spectrum. In the case of sampling your $5\mathrm{kHz}$ wave at $4\mathrm{kHz}$, you have real honest-to-gosh sine waves at 4, 6, 9, 11, 14, 16, etc.. These constructively or destrictively interfere, making the strange shape you see.

If you sample at $6\mathrm{kHz}$ and reconstruct, you see those same frequencies, because of aliasing.

If you took that $4\mathrm{kHz}$ reconstructed sine wave and ran it through a low-pass filter that passed everything up to and including $4\mathrm{kHz}$ and nothing at or above $6\mathrm{kHz}$, then you'd get back a perfect reconstruction. If you sampled a $4.99\mathrm{kHz}$ wave, sampled it and reconstructed it, then you could get a perfect reconstruction, too, but now you'd need a filter that passes everything up to and including $4.99\mathrm{kHz}$ and nothing at or above $5.01\mathrm{kHz}$.

The difficulty arises because the narrower the filter you use, the harder it is to realize physically and the more delay you need. In theory you can start with a signal at $f = F_s - \epsilon$, and for any $\epsilon > 0$ the sampled signal can be reconstructed -- but the closer $\epsilon$ gets to zero, the harder things get.

In practice, you'd like to use $f < F_s/2$ or even less -- and there's good solid arguments why, for audiophile signals, you want it four or eight times less. Things like compact disks use less of a ratio than this -- but they need to do lots of fancy filtering to make it work, and there are audiophiles who are unhappy with the CD sampling rate of 44-odd kHz.

* Note that I'm purposely avoiding the most popular model of sampling that's used in DSP, where a sampled-time signal is multiplied by a train of Dirac delta functions. This is so I can fit the discussion into one reply with at least some rigor. If you dive into DSP for real, just trust me that in the end the Dirac delta model is better -- it's just more work to wrap your brain around at first.

Answer 3

You're right, in the real world, 2x is not enough to capture a sound at a given frequency accurately. In your chart, only the first one would sound like a sine wave. As you approach the Nyquist frequency, you'll create a siren like sound and when you reach the exact frequency you'll record a pulse-wave approximation of a sine wave at an amplitude that will vary based on your phase. There is no reconstruction DAC that can take a series of zeroes and make a sine wave out of that unless you're introducing hidden data.

Even at 4x, you've only reconstructed a triangle wave and if you shift your phase, you'll get some tonal distortion. Using a cosine or cubic interpolator on the DAC, you need at least a 6x sample rate to accurately reconstruct the sine wave making NO ASSUMPTION of what the original sample does. Nyquist only works in a very unreal example case and it's not actually useful to audio engineers.

"What the Nyquist-Shannon sampling theorem—absolutely and positively— does not say, is that you can design your system to operate right at the Nyquist rate, at least not with any reasonable chance of success."

https://www.wescottdesign.com/articles/Sampling/sampling.pdf