Limit $f/f_s \to 0$
===================
Let the zero-to-peak amplitude of the continuous-time *prototypical sinusoid*, which is being sampled and quantized, be an integer $A$. In the limit $f/f_s \to 0,$ where $f$ is the sine wave frequency and $f_s$ is the sampling frequency, the amplitudes of the harmonics are found in the [Fourier series][1] representation of the continuous-time piece-wise constant quantized waveform. The terms of the Fourier series are directly the harmonics of the fundamental frequency. We can construct the waveform as a sum of components like the red curve here, illustrated for sine wave amplitude $A = 7$:
[![enter image description here][2]][2]<br>*Figure 1. prototypical sinusoid, of amplitude $A=7$, and its quantization in blue, with a component highlighted in red.*
We are working with cosine instead of a sine because the math is nicer this way. Because of the symmetries, each component has only odd harmonics and only cosine terms in its Fourier series. The non-zero coefficients of the Fourier series are given by:
$$a_n = \frac{4}{\pi}\int_{x_0}^{x_1}\cos(nx)\,k\,dx = \frac{4k}{\pi n}\big(\sin(x_1n) - \sin(x_0n)\big),\quad n\text{ odd},\tag{1}$$
with integer amplitude $k \in 1\ldots A$ in range $x_0 < x < x_1$ in the first quarter-period of the cosine. I did not bother to explicitly write the contributions from the symmetrically arranged pieces in the other quarters, because they will contribute identically to the odd harmonic cosine terms in the Fourier series. Instead I simply included the implicit factor 4 in the equation.
The Fourier series of the full piece-wise constant waveform is a sum of the Fourier series of the components. The boundaries of the pieces that we need to include in the sum are:
$$\begin{gather}0 < x < \operatorname{acos}\left(\frac{A - 0.5}{A}\right),\quad \text{if }k = A,\\
\operatorname{acos}\left(\frac{k + 0.5}{A}\right) < x < \operatorname{acos}\left(\frac{k - 0.5}{A}\right),\quad \text{if }k \in 1\ldots A-1,\end{gather}\tag{2}$$
where $\operatorname{acos}\left(\frac{s}{A}\right)$ comes from solving $A\cos(x) = s$ in the first quadrant. The non-zero coefficients of the Fourier series of the full piece-wise constant waveform are then given by:
$$\begin{gather}a_n = \frac{4A}{\pi n}\sin\bigg(\operatorname{acos}\left(\tfrac{A - 0.5}{A}\right)n\bigg) +\\\frac{4}{\pi n}\sum_{k=1}^{A-1} \Bigg(k\sin\bigg(\operatorname{acos}\left(\tfrac{k - 0.5}{A}\right)n\bigg) - k\sin\bigg(\operatorname{acos}\left(\tfrac{k + 0.5}{A}\right)n\bigg)\Bigg),\quad n\text{ odd}.\end{gather}\tag{3}$$
16-bit quantization
-------------------
It is not out of reach to compute Eq. 3 for $A=2^{23} - 1$ and $n \in {5, 7},$ as parameterized in the question, using something like Python's `mpmath`. Let's try that but with a 16-bit sine wave first:
import mpmath as mp
def a_n(A, n):
if int(n) & 1:
return 4*A/(mp.pi*n)*mp.sin(mp.acos((A-0.5)/A)*n) + 4/(mp.pi*n)*mp.nsum(lambda k: k*( mp.sin(mp.acos((k-0.5)/A)*n) - mp.sin(mp.acos((k+0.5)/A)*n) ), [1, A-1])
else:
return 0
A = 2**15-1 # amplitude (integer)
n = 5 # number of the harmonic (odd integer)
mp.mp.prec = 53 # default precision
20*mp.log10(mp.fabs(a_n(A, n))/A) # Fourier series term amplitude in dB
mp.mp.prec = 106 # Compute again with double the default precision, see if we get the same result
20*mp.log10(mp.fabs(a_n(A, n))/A) # Fourier series term amplitude in dB
mp.mp.prec = 53 # Restore default precision
I'm only reporting as many digits as are agreed about by repeated computation using two different `mpmath` precision settings, including also the digits that got changed by $\pm1$ due to larger changes in later digits. For a 16-bit full-scale $A = 2^{15}-1$ sine wave and $f/f_s \to 0$, the result is **-154.66449 dBFS** for the 5th harmonic and, after setting `n = 7` in the script, **-154.66506 dBFS** for the 7th harmonic, in about 15 seconds of computation on my PC for a single result with the higher precision setting. As a sanity check, I also computed the amplitude of the fundamental frequency (1st harmonic), which is interestingly 0.0000001605530 dBFS (or 1.605530E-7 dBFS in scientific notation) compared to the peak value $2^{15}-1$ at 0 dBFS. I think the rounding at the top of the sine wave is "pulling it up", giving it a higher amplitude of the fundamental.
Let's have a closer look at the decay of the amplitudes of the odd harmonics:
for m in range(1, 20):
n = mp.mpf(10)**m+1 # number of the harmonic
mp.mp.prec = 70 # precision
a_n_low_prec = a_n(A, n)
mp.mp.prec = 90 # precision
a_n_high_prec = a_n(A, n)
print(str(n)+","+str(20*mp.log10(mp.fabs(a_n_low_prec)/A))+","+str(20*mp.log10(mp.fabs(a_n_high_prec)/A)))
[![enter image description here][3]][3]<br>*Figure 2. Amplitudes of a selection of harmonics of the quantized prototypical sinusoid, with $A=2^{15}-1$.*
The graph in Fig. 2 is not very smooth, and it would probably look more jagged if we were to plot all points. I think this is because the waveform is similar to a sawtooth riding on a sine wave, and near the zero crossing the sawtooth frequency stays the same over a large portion of the waveform. Actually, the sawtooth frequency is $2\pi A$ times the fundamental frequency, which approximately equals that of the 2.05881E5th harmonic. Indeed, the amplitude of that harmonic turns out as -135.49405259 dBFS, seemingly much higher than that of a "random" early harmonic. It would also seem that the amplitudes of the harmonics start to roll off after this frequency, with the factor $1/n$ in Eq. 3 making the asymptotic decay -20 dB/decade (~-6 dB/octave).
24-bit quantization
-------------------
The following continuation of the earlier Python script computes in high precision your desired numbers for $A=2^{23} - 1$, in the limiting case $f/f_s \to 0$:
A = 2**23-1 # amplitude (integer)
n = 5 # number of the harmonic (odd integer)
mp.mp.prec = 70 # Precision 1
20*mp.log10(mp.fabs(a_n(A, n))/A) # in dBFS
mp.mp.prec = 140 # Precision 2
20*mp.log10(mp.fabs(a_n(A, n))/A) # in dBFS
n = 7 # number of the harmonic (odd integer)
mp.mp.prec = 70 # Precision 1
20*mp.log10(mp.fabs(a_n(A, n))/A) # in dB
mp.mp.prec = 140 # Precision 2
20*mp.log10(mp.fabs(a_n(A, n))/A) # in dB
mp.mp.prec = 53 # Restore default precision
After a total of a few hours of computation on my PC, the result is that with $A=2^{23} - 1$ and $f/f_s \to 0,$ the amplitude of the 5th harmonic is **-226.91150085 dBFS** and the amplitude of the 7th harmonic is **-226.9115030 dBFS**. I also computed the amplitude of the fundamental frequency, 3.9195785E-11 dBFS, and in a full-day computation the amplitudes of a number of higher harmonics, in Python:
A = 2**23-1 # amplitude (integer)
for m in range(1, 20):
n = mp.mpf(10)**m+1 # number of the harmonic
mp.mp.prec = 70 # precision
a_n_low_prec = a_n(A, n)
mp.mp.prec = 90 # precision
a_n_high_prec = a_n(A, n)
print(str(n)+","+str(20*mp.log10(mp.fabs(a_n_low_prec)/A))+","+str(20*mp.log10(mp.fabs(a_n_high_prec)/A)))
[![!\[enter image description here][4]][4]<br>*Figure 3. Amplitudes of a selection of harmonics of the quantized prototypical sinusoid, with $A=2^{23}-1$.*
The corner frequency before the roll-off is again approximately the sawtooth frequency at the zero crossing of the quantized prototypical sinusoid, $2\pi A = 2\pi(2^{23}-1) \approx$ 5.270717225E7 times the fundamental frequency $f$.
Rational $f/f_s$
================
Let the prototypical sinusoid, which is being sampled and quantized, be a cosine without a phase shift. That is to say, one of the samples is at the peak of the sinusoid. Let $f/f_s$ be equal to a rational number $c/d$, with $c$ and $d$ integer. The fundamental period $L$ of the [periodic sequence][5] of quantized samples is:
$$L = \frac{d}{\operatorname{gcd}(c, d)},\tag{4}$$
where gcd denotes the greatest common divisor. A real discrete Fourier transform (real-DFT) of length $L$ has bin frequencies that are the harmonic frequencies of the period-$L$ frequency. The period-$L$ frequency is equivalent to the frequency of the prototypical sinusoid only if the numerator of the [irreducible fraction][6] representation of $f/f_s$ equals 1. Otherwise the real-DFT bin frequencies contain also other frequencies than those equivalent to in-band harmonics of the sinusoid.
What we have learned from the results so far is that the amplitudes of the harmonics of the quantized prototypical sinusoid do not really start to decay until after the approximately $2\pi A$th harmonic. This means that with rational $f/f_s$, the amplitude of an in-band harmonic frequency of the quantized sequence is typically almost fully determined by aliased frequencies. As an example of such aliasing, with $f_s =$ 48 kHz and $f =$ 1000 Hz, the 5th harmonic gets summed with the aliases of the 43th, 53th, 91th, 101th, 139th, 149th, etc. harmonic:
$$\begin{eqnarray}
&& \ldots\\
&=& 48000\text{ Hz}\times -3 + 1000\text{ Hz}\times 149\\
&=& 48000\text{ Hz}\times -2 + 1000\text{ Hz}\times 101\\
&=& 48000\text{ Hz}\times -1 + 1000\text{ Hz}\times 53\\
&=& 1000\text{ Hz}\times 5\\
&=& 48000\text{ Hz}\times 1 - 1000\text{ Hz}\times 43\\
&=& 48000\text{ Hz}\times 2 - 1000\text{ Hz}\times 91\\
&=& 48000\text{ Hz}\times 3 - 1000\text{ Hz}\times 139\\
&=& \ldots\end{eqnarray}\tag{5}$$
The effect of summation of the harmonics aliased to a frequency depends also on the phase of the sinusoid being quantized, because the phases of the aliased harmonics control whether there is constructive or destructive interference. Aliasing does not change the phase of a harmonic. A phase shift of the quantized prototypical sinusoid is equivalent to shifting it in time, which accounts to a phase shift of each unaliased harmonic proportional to its number. In this answer we will only be sampling and quantizing a prototypical cosine without a phase shift, so to compute a single bin of the real-DFT of the discrete-time periodic sequence of samples, it would suffices to sum those $a$ coefficients of the Fourier series of the quantized prototypical sinusoid that alias to the equivalent frequency of the bin. If the numerator of $f/f_s$ expressed as an [irreducible fraction][6] equals 1, we have:
$$\operatorname{bin}_n = a_n + \sum_{k=1}^{\infty}\left(a_{kL-n} + a_{kL+n}\right),\tag{6}$$
where $a_n$ is given by Eq. 3 for odd $n$ and $a_n$ is zero for even $n$. With $f = $ 1000 Hz, $f_s = $ 48 kHz, on my PC it would take perhaps thousands of years to compute Eq. 6 to reasonable accuracy for $A=2^{23}-1$, and perhaps weeks for $A=2^{15}-1$. I also have difficulties evaluating the series even with $A=2^3-1$, with the sequence of partial sums oscillating. I think this difficulty arises from that with $A = 2^3 - 1$ and $f/f_s = 1/48$, the sequence before quantization contains values $-3.5$ and $3.5$ exactly at a discontinuity in the quantized prototypical sinusoid. In any case, with rational $f/f_s$, the DFT-based approach to determine the amplitudes of the harmonics is much more practical.
24-bit quantization
-------------------
The following Python script computes using the DFT-based method the amplitudes of the harmonics for $A = 2^{23}-1$, $f =$ 1000 Hz, and $f_s =$ 48 kHz:
import numpy as np
from sympy import Rational
def bins(A, c, d):
L = Rational(d, np.gcd(c, d)) # Fundamental period (samples)
waveform = np.around(A*np.cos(np.dot(range(L), 2*np.pi*c/d)))
return np.fft.rfft(waveform)/(A*L/2)
A = 2**23-1 # amplitude (integer)
c = 1000 # numerator of f/f_s (positive integer)
d = 48000 # denominator of f/f_s (positive integer)
[20*np.log10(float(abs(x))) for x in bins(A, c, d)] # real-DFT magnitudes (dBFS)
While not required by the script, in this case $c/d = 1/48,$ so each real-DFT bin corresponds to a harmonic of the prototypical sinusoid. The fundamental frequency's amplitude measures as -1.75E-7 dBFS, the 5th harmonic as **-160.90 dBFS** and the 7th as **-160.75 dBFS**. Unfortunately I could not find a suitable multiple precision Fast Fourier Transform (FFT) library, but I think those three numbers came out in sufficient precision using NumPy's `rfft`. The precision is not sufficient to calculate the amplitude of say the 3rd harmonic at about -333 dBFS (Fig. 4).
[![enter image description here][7]][7]<br>*Figure 4. Magnitudes of the real-DFT bins of the quantized sequence with $A=2^{23}-1$, $f =$ 1000 Hz, and $f_s =$ 48000 Hz.*
Validation
----------
We can validate the two approaches against each other by choosing a pair of $A$ and $f/f_s$ that does not result in sampling of the quantized prototypical sinusoid at any of its discontinuities. Firstly, this removes having to choose arbitrarily which way a number exactly half-way between integers should be rounded, and secondly, should help in convergence of the infinite series in Eq. 6. Let's try $A = 8$ and $f/f_s = 1/48$. First, the DFT-based method, in Python:
import numpy as np
from sympy import Rational
def bins(A, c, d):
L = Rational(d, np.gcd(c, d)) # Fundamental period (samples)
waveform = np.around(A*np.cos(np.dot(range(L), 2*np.pi*c/d)))
return np.fft.rfft(waveform)/(A*L/2)
A = 8 # amplitude (integer)
c = 1000 # numerator of f/f_s (positive integer)
d = 48000 # denominator of f/f_s (positive integer)
[20*np.log10(float(abs(x))) for x in bins(A, c, d)] # real-DFT magnitudes (dBFS)
This results in harmonic amplitudes, printed in precision beyond their accuracy, in dBFS:
-inf, 0.005622747208892056, -inf, -331.19944825653926, -inf, -50.41376796795221, -inf, -35.41599672115829, -inf, -327.2878768807577, -inf, -38.14783193548751, -inf, -45.42857299685606, -inf, -324.8188873214554, -inf, -40.41625528549065, -inf, -36.12873038111221, -inf, -337.55682735433885, -inf, -33.04816436790989, -inf
Then, aliasing the harmonics of the quantized prototypical cosine by evaluating Eq. 6, with the series truncated to the first approximately $2000\pi A$ terms, in Python:
import mpmath as mp
def a_n(A, n):
if int(n) & 1:
return 4*A/(mp.pi*n)*mp.sin(mp.acos((A-0.5)/A)*n) + 4/(mp.pi*n)*mp.nsum(lambda k: k*( mp.sin(mp.acos((k-0.5)/A)*n) - mp.sin(mp.acos((k+0.5)/A)*n) ), [1, A-1])
else:
return 0
mp.mp.prec = 70
L = Rational(d, np.gcd(c, d)) # Fundamental period (samples)
for n in range(L/2):
bin = a_n(A, n) + mp.nsum(lambda k: a_n(A, k*L - n) + a_n(A, k*L + n), [1, int(2000*mp.pi*A)])
float(20*mp.log10(mp.fabs(bin)/A)) # harmonic amplitude (dBFS)
mp.mp.prec = 53
which results in amplitudes, printed in precision beyond their accuracy, in dBFS:
-inf, 0.00562340858138011, -inf, -141.4195064523128, -inf, -50.413977325520634, -inf, ...
It looks like the results may agree within their numerical accuracy, but it is not certain. I'll try to improve the accuracy and continue the computation, to see if the results can be made to match better.
[1]: https://en.wikipedia.org/wiki/Fourier_series
[2]: https://i.sstatic.net/1a4gD.png
[3]: https://i.sstatic.net/opb6G.png
[4]: https://i.sstatic.net/Gh3Jt.png
[5]: https://en.wikipedia.org/wiki/Periodic_sequence
[6]: https://en.wikipedia.org/wiki/Irreducible_fraction
[7]: https://i.sstatic.net/npZyu.png