Why Hilbert Transform is terrible choice for amplitude demodulation of broadband signals?

Question

A reference answer empirically demonstrates that Hilbert envelope does not work well for the (amplitude) demodulation of a broadband signal. I am looking for the math which explains why...

Answer 1

It's not only a matter of "broadband or not":

The Hilbert estimate degrades for multi-component signals - that is, whatever we can't draw without lifting our pen, left-to-right, in time-frequency (STFT), because then, we have multiple amplitudes to track:

$$ A_1(t) \cos(\phi_1 (t)) + A_2(t) \cos(\phi_2 (t)) $$

and there's no way of "combining" $A_1$ and $A_2$ for the desired effect. This isn't to suggest that a satisfactory $A$ doesn't exist, only that Hilbert won't yield it: the Hilbert transform yields the analytic representation, $A(t)\cdot e^{j\phi (t)}$, whose real part is the original signal, and the "envelope" is $A(t)$, which happens to match the signal's $A$ only if the signal itself can be written as $A \cdot \cos$. And more precisely, it's $|A(t)|$, but in practice $A(t) \geq 0$ anyway.

It just happens so that, for real-world broadband signals, it's likelier to have multiple components. The degradation can very well occur within narrowband signals also, but it's less likely. Explained further in synchrosqueezing (includes math on what $A$ and $\phi$ can be, to not be fully arbitrary).

Extended note

General amplitude extraction isn't something a simple 1D transform like Hilbert can handle. It is intrinsically tied to time-frequency resolution and subject to Heisenberg's uncertainty. In my expressions above, $A$ and $\phi$ must satisfy certain criteria for any method to work, plus the criteria I stated for Hilbert transform to work. Most importantly, the highest significant (high energy) frequency of $A(t)$ must be much smaller than the lowest significant frequency of $\cos(\phi(t))$:

If the carrier is a single pure sine, "much smaller than" relaxes to "strictly less than" (there's dependence on $\phi''$, see reference). In Gideon's derivation, $A(y(t))=x(t)e^{−j\omega_c t}$ is only true if the highest frequency of $x(t)$ is less than $\omega_c$ - and in the single pure sine carrier case, it'll work even if $x(t)$ is multi-component.

See my other answer for a proof of these claims.

Answer 2

Amplitude extraction / AM demodulation criteria

I shall prove,

1. $y(t) = x(t) \cos(\omega_c t)$ demodulates perfectly to $|x(t)|$ if
   • A) $x$'s highest frequency, $\omega^\text{max}_x$, is $<\omega_c$
   • B) $\omega^\text{max}_x + \omega_c < \omega_s/2$ (sub-Nyquist)
2. $y(t) = \cos(\omega_m t) \cos(\omega_c t)$ always demodulates to $|\cos(\text{min}(\omega_m, \omega_c)t)|$.

1 says "pure sine can carry any message whose highest frequency is below its own". It will also be shown, a pure sine is the only carrier which can perfectly demodulate.

2 says "the envelope is always the lesser frequency in a product of sines".

Both are for the discrete setting. Implications for STFT and CWT are described.

Criteria notes

• "can demodulate" in this post refers only to "directly from analytic signal" (without further steps)
• "pure sine" includes complex sinusoid
• All $\omega$ in discrete/Nyquist context includes aliased frequencies; Ctrl + F "Alias note"
• This answer is mainly about real-valued $y(t)$, but many conclusions carry directly and others with appropriate adjustment

Notation / identities

• Analytic signal: $\mathcal{A}\{y(t)\} = y(t) + j \mathcal{H}\{y(t)\}$
• Hilbert transform: $\mathcal{H}$
• product $\Leftrightarrow$ sum: $\cos(A) \cos(B) = .5 (\cos(A + B) + \cos(A - B))$
• "Demodulation", "amplitude extraction", and "envelope extraction" are all used interchangeably and refer to the same thing: recovering $|x|$ from $\mathcal{A}$ via $|\mathcal{A}|$.

Proof of 2

We know $\mathcal{H}\{\cos(\omega t)\} = \sin(|\omega| t)$, hence (assume $\omega \geq 0$ for now)

$$ \mathcal{A}\{\cos(\omega t)\} = \cos(\omega t) + j\sin(\omega t) = e^{j\omega t} $$

A minimal AM-carrier is

$$ \cos(\omega_m t) \cos(\omega_c t), $$

and we know (ignore $.5$) that's just

$$ \cos((\omega_m + \omega_c)t) + \cos((\omega_m - \omega_c)t). $$

Simplify notation, take $A=\omega_m t$ and $B=\omega_c t$. By linearity of Hilbert,

$$ \mathcal{H}\{\cos(A + B) + \cos(A - B)\} = \sin(A + B) + \sin(|A - B|) $$

Then for modulus, $|\mathcal{A}\{\cos(A + B) + \cos(A - B)\}|$, we have

$$ \sqrt{(\cos(A + B) + \cos(A - B))^2 + (\sin(A + B) + \sin(|A - B|))^2} $$

and one can show that's

$$ \begin{cases} 2|\cos(B)|, & A > B\\ 2|\cos(A)|, & \text{otherwise} \end{cases} $$

which is same as

$$ 2|\cos(\text{min}(A, B))|. $$

In original notation, this reads (now don't ignore .5)

$$ \boxed{ |\mathcal{A}\{ \cos(\omega_m t)\cos(\omega_c t) \}| = |\cos(\text{min}(\omega_m, \omega_c))| } $$

so whether we like it or not, Hilbert considers the lesser frequency the "modulator". To preserve the "message", we should ensure the envelope is additionally $>0$ so the modulus on the right side has no effect - easily accomplished by adding a constant, which we can then subtract.

Small caveat, the $A, B$ notation conceals the fact that the derived identity requires $\omega_c + \omega_m \geq 0$ (see result), but that's a non-issue. Lastly, one can redo the derivation with $\omega<0$, and sines instead of cosines, to obtain the same result (which covers every possible sign combination, hence proving it's a non-issue).

Proof of 1

Suppose the message is $\cos(\omega_{m_0}t) + \cos(\omega_{m_1}t)$, so we have

$$ \left(\cos(\omega_{m_0}t) + \cos(\omega_{m_1}t)\right) \cos(\omega_c t). $$

We know that's (ignore $.5$)

$$ [\cos((\omega_{m_0} + \omega_c)t) + \cos((\omega_{m_0} - \omega_c)t)] + \\ [\cos((\omega_{m_1} + \omega_c)t) + \cos((\omega_{m_1} - \omega_c)t)] \phantom{+.} $$

By linearity of Hilbert, we have above but with sines (and modulus around the $-$). Simplifying notation again,

$$ \begin{align} \mathcal{A}\{\cdot\} = & [\cos(A + C) + \cos(A - C) + \cos(B + C) + \cos(B - C)] + \\ j & [\sin(A + C) + \sin(|A - C|) + \sin(B + C) + \sin(|B - C|)] \end{align} $$

Now, we conjure a trick; we know $C > A$ from our assumption that $\omega_c$ is the greatest. Then,

$$ \sin(C - A) = \sin(|A - C|), \ \ \text{if}\ C > A $$

and we rewrite the imaginary part as

$$ \sin(C + A) + \sin(C - A) + \sin(C + B) + \sin(C - B) $$

We know

$$ \sin(a + b) + \sin(a - b) = 2\sin(a)\cos(b), $$

and hence,

$$ 2\cdot (\sin(C)\cos(A) + \sin(C)\cos(B)). $$

Together with reals, we factor and obtain

$$ \begin{align} \mathcal{A}\{\cdot\} = & 2\cos(C)[\cos(A) + \cos(B)] + \\ j & 2\sin(C)[\cos(A) + \cos(B)] \end{align} $$

which under modulus is

$$ \sqrt{4\cos^2(C)(\cos(A) + \cos(B))^2 + 4\sin^2(C)(\cos(A) + \cos(B))^2} = \\ \sqrt{4(\sin^2(C) + \cos^2(C))(\cos(A) + \cos(B))^2} = \\ 2 |\cos(A) + \cos(B)|. $$

Retrieving our ignored $.5$ and reverting notation, we thus have

$$ |\mathcal{A}\{\left(\cos(\omega_{m_0}t) + \cos(\omega_{m_1}t)\right) \cos(\omega_c t))\}| = |\cos(\omega_{m_0}t) + \cos(\omega_{m_1}t)|. $$

Lastly, we notice that the real and imaginary parts under modulus, are linear in addition - so adding $\omega_{m_2}$, $\omega_{m_3}$, etc will not introduce dependence on $\omega_c$. Furthermore, the $\sin(C)$ will still factor with the modulators having unequal amplitudes:

$$ 2\cos(\omega_{m_0}t) + 3\cos(\omega_{m_1}t) $$

would yield, for the earlier imaginary part,

$$ 2\cdot (2\sin(C)\cos(A) + 3\sin(C)\cos(B)). $$

Thus, the general result (for any $K$ number of modulators) is

$$ \boxed{\left|\mathcal{A}\{\cos(\omega_c t) \sum_{i=0}^{K-1}A_i\cos(\omega_{m_i}t) )\}\right| = \left| \sum_{i=0}^{K-1}A_i\cos(\omega_{m_i}t) \right|}. $$

Notice, this summation is identically the Discrete Fourier Transform, except with cosines instead of cisoids. However, we note that the DFT, for a real input, is identically a summation of real cosines and sines, as the imaginary part must sum to zero.

If we can show that the modulus remains independent of $C$ even with sines, then we've proven what we sought, since as the sum on right hand side becomes a DFT equivalent, which can build any messenger $x(t)$. Put differently, any $x(t)$ we pick is possible to rewrite as that sum.

Proof of 1, part 2

Take

$$ \left(\cos(\omega_{m_0}t) + \sin(\omega_{m_1}t)\right) \cos(\omega_c t). $$

Simplify notation ahead of time. We know that's (ignore $.5$)

$$ [\cos((\omega_{m_0} + \omega_c)t) + \cos((\omega_{m_0} - \omega_c)t)] + \\ [\sin((\omega_{m_1} + \omega_c)t) + \sin((\omega_{m_1} - \omega_c)t)] \phantom{+.} $$

and $\mathcal{H}\{\sin(\omega t)\} = -\text{sgn}(\omega)\cos(\omega t)$. Suppose again the individual $\omega$'s are non-negative; then the $+$ stay negated, but $(\omega_{m_1} - \omega_c)$ double-negates into $+$. Hence,

$$ \begin{align} \mathcal{H}\{\cdot\} = & [\sin((\omega_{m_0} + \omega_c)t) + \sin(|\omega_{m_0} - \omega_c|t)] + \\ & [-\cos((\omega_{m_1} + \omega_c)t) + \cos((\omega_{m_1} - \omega_c)t)] \phantom{+.} \end{align} $$

and, simplifying notation, we obtain

$$ \begin{align} \mathcal{A}\{\cdot\} = & [\cos(A + C) + \cos(A - C) + \sin(B + C) + \sin(B - C)] + \\ j & [\sin(A + C) + \sin(|A - C|) - \cos(B + C) + \cos(B - C)] \end{align} $$

Now we apply

$$ \cos(a - b) - \cos(a + b) = 2\sin(a)\sin(b) $$

hence

$$ 2\sin(B)\sin(C) $$

Repeating our arguments from part 1, we again factor $\sin(C)$ from the sines in the imaginary part, and $\cos(C)$ from the real part, and in total obtain

$$ \begin{align} \mathcal{A}\{\cdot\} = & 2\cos(C)[\cos(A) + \sin(B)] + \\ j & 2\sin(C)[\cos(A) + \sin(B)] \end{align} $$

which under modulus is

$$ \sqrt{4\cos^2(C)(\cos(A) + \sin(B))^2 + 4\sin^2(C)(\cos(A) + \sin(B))^2} = \\ \sqrt{4(\sin^2(C) + \cos^2(C))(\cos(A) + \sin(B))^2} = \\ 2 |\cos(A) + \sin(B)|. $$

Repeating the rest of the arguments from part 1, and additionally observing that we did not make $\omega_{m_1}$ depend on $\omega_{m_0}$ in any way and hence they can be equal, and that we can have $A_i=0$ while $B_i \neq 0$ and vice versa, we have thus shown,

$$ \boxed{ \left|\mathcal{A} \{ \cos(\omega_c t) \sum_{i=0}^{K-1}\left(A_i\cos(\omega_{m_i}t) + B_i\sin(\omega_{m_i}t) \vphantom{\frac{.}{.}}\right) \} \right| = \left| \sum_{i=0}^{K-1}\left(A_i\cos(\omega_{m_i}t) + B_i\sin(\omega_{m_i}t) \vphantom{\frac{.}{.}}\right) \right|} $$

Finally, to prove my claim that real DFT is just sum of sines and cosines - observe, irfft does, for any bin $k$,

$$ (A_k + jB_k)\cdot (\cos(2\pi k n) + j\sin(2\pi k n)) \\ [A_k\cos(2\pi k n) - B_k\sin(2\pi k n)] + j[A_k\sin(2\pi k n) + B_k\cos(2\pi k n)] $$

and the imaginary part must sum to zero, leaving

$$ A_k\cos(2\pi k n) - B_k\sin(2\pi k n) $$

which is

$$ A_i\cos(\omega_{m_i} t) + B_i\sin(\omega_{m_i} t) $$

in disguise.

Remark: discrete vs continuous

All identities used are pointwise relations, hence regime-agnostic. The two critical differences are,

1. Existence of Nyquist for discrete; note that $\cos(A)\cos(B)$ contains $\cos(A + B)$, so the sum of carrier and messenger's peak frequencies cannot exceed (or reach) half of sampling rate.
2. What qualifies as "pure sine"; in discrete time, this means integer number of cycles. This yields a single DFT bin, otherwise, we get "spectral leakage" - frequencies above and below $\omega_c$, which violates our core assumption, and also makes the carrier multi-component as far as the proof (and DFT) is concerned (or, to be precise, the discrete Hilbert transform is no longer the sampling of the continuous Hilbert transform of an infinite pure sine, just like the DFT of such a sine is no longer coincides with sampling of a pure sine's CFT via DTFT).

The messenger will be as well-preserved as our satisfying of above. It's also another reason we favor high frequency carriers: the difference between exact number of integer cycles and not, is much fewer in samples, so our error is smaller.

As to whether this proof holds in continuous-time... dunno. This proof uses the convenience of trig identities thanks to sums - in CT we have integrals. The relations continue to hold for arbitrarily large $N$, so perhaps we argue, in Riemannian spirit, that the continuous case is also proven via $N\rightarrow \infty$.

Proof of 1, code

import numpy as np
from scipy.signal import hilbert

for N in (128, 129):  # even & odd cases
    t = np.linspace(0, 1, N, 0)
    for f_c in range(N//4):  # //4 to avoid alias per *amplitude
        # make general `x` with freqs all the way up to (but excluding) `f_c`
        x_f = np.random.randn(N) + 1j*np.random.randn(N)
        # same as zeroing between +f_c and -f_c, w/ + reflected to - for Herm. symmetry
        x_f[f_c:] = 0
        x = np.fft.ifft(x_f).real
    
        c = np.cos(2*np.pi * f_c * t)
        y = x * c
    
        assert np.allclose(np.abs(x), np.abs(hilbert(y)))

Proof: multi-component

To prove the envelope won't extract in presence of multiple carriers, we simply make $\omega_c$ the smallest frequency in above derivations, which is exactly what we'd write for two carriers modulated by the same envelope (and invoke proof 2). Then, we can no longer use $\sin(|A-C|) = \sin(C-A)$, which breaks our ability to factor out and eliminate $\omega_c$, thus retaining variable dependence on $\omega_c$.

Conclusion

This completely disproves the notion that bandwidth alone determines ability to demodulate, as our signal can span everything from DC to Nyquist with maximum energy and still demodulate.

The multi-component case requires time-frequency analysis, since the decomposition into "carriers" and "modulators" is non-unique, and component separability is intrinsically bound by time-frequency resolution and Heisenberg's principle. See my other answer.

STFT, CWT implications

STFT and analytic CWT are simply bandpassed analytic signals. Hence any conclusions drawn for the general analytic signal carry over. Most notably, neither can perfectly extract the amplitude of anything but an AM pure sine: multi-components and AM-FMs do not perfectly demodulate via $|\mathcal{A}|$, and linear filtering cannot change that. Synchrosqueezing, however, can: the reassignment step is nonlinear and can reduce every temporal slice to a single point, which via invertibility guarantees perfect recovery.

Observations

"demodulation", "envelope extraction", "amplitude extraction" below all refer to obtaining $|x(t)|$ via $|\mathcal{A}|$ (without further steps). I use "demodulation" for brevity. "message" is $x(t)$.

1. Exact demodulation is only possible with a pure sine carrier. Follows directly from the multi-component proof, where any frequency that's not the pure sine will qualify as "component".

2. Exact demodulation is only possible with messages whose maximum (nonzero) frequency is below half of Nyquist, i.e. $< f_s/4$. The worst case is max frequency of $f_s/4 - 1$, for which the best possible carrier is $f_s/4$, that results in populating $x(t)c(t)$ all the way up to Nyquist (see next point). Else, alias.

   • Another motivation to DC offset our message, since otherwise $|x|$'s max freq is typically unbounded (even for a pure sine), while ensuring $x \geq 0$ yields $x=|x|$.
   • Note, per previous distinction, an unbounded $|\mathcal{A}\{y(t)\}|$ doesn't mean an unbounded $x$ and violation of any criteria. But it does mean, if we expect $x \geq 0$ (e.g. if $x$ is supposed to be the amplitude function of some AM-FM signal), then we aren't in clear and $x$ isn't recovered.

3. Spectrum of $x(t)c(t)$ is simply spectrum of $x$ shifted to $c$'s bin; suppose its freq is $f_s/4$:

$$ x(t)c(t) \Leftrightarrow X(f) \star C(f) = X(f) \star \delta(|f| - f_s/4) = X(|f| - f_s/4) $$

from this we get another perspective on why $\omega_m < \omega_c$ must hold: if it doesn't, then fft(x), when shifted, overlaps itself and aliases.

4. Exact demodulation only occurs if $y$'s spectrum is even-symmetric about some bin, separately (but identically) for positive and negative frequencies. Follows 1, 2, and 3. However, this is a necessary but not sufficient criterion: possible to have such symmetry but not recover original $x$ (e.g. carrier is two pure sines of equal amplitude in adjacent bins).

5. The "necessary and sufficient" version of 4 additionally requires that $y$'s spectrum have an odd number of non-zero samples, and that the central sample is greater than or equal to the sum of absolute values of all other samples. Central sample = DC (can have only one), and worst case to guarantee $|x| = x$ is $X[k=0] = \sum |X[k\neq 0]|$, which is much greater than what's required in practice ($X[0]=|\text{min}(x)|$). Note (4) doesn't aim for $x=|x|$, but this "necessary and sufficient" guarantees $x=|x|$.

6. $\text{bw}(x) \leq \text{bw}(y)/2$, assuming no alias ($/2$ i.e. $y$'s one-sided spectrum). Follows from 3 and linearity: the best case is a pure sine carrier, where $\text{bw}(x) = \text{bw}(y)/2$ - anything else and we're adding more shiftings of fft(x), thus expanding fft(y). Also hence, multi-component carriers will alias fft(x) (per overlaps, and not to be confused with aliasing $y$).

7. $\text{bw}(|\mathcal{A}\{y(t)\}|)$ for multi-component carrier is almost always unbounded: note when we don't cancel $C$ in the derivations, we get an irreducible sum of sines and cosines under the square root. In fact it may be possible to prove that the multi-component case is always unbounded. At the same time, it can certainly be "close enough to" bounded, as is the case with the modulus of STFT and analytic CWT.

8. (Alias note) Infinite valid carriers and message frequency shifts: $\omega_c$ and the rest, in a discrete setting, include the continuous $\tilde \omega_c$ that become $\omega_c$ after sampling. For example, if $f_s = 200\text{Hz}$, then $\tilde f_c = 300, 500, 700, ... \text{Hz}$ become $f_c = 100 \text{Hz}$ after sampling. The neat thing is, there's infinite qualifying continuous carriers and shifts of $\hat x(t)$, but bandwidth of the message must still remain sub-Nyquist.

Answer 3

Assume that we have a signal $$x(t)=\cos(\omega_1 t) + \cos(\omega_2) + 3$$ with a carrier $$c(t) = \cos(\omega_c t)$$ that is not known apriori.

Than, for the amplitude modulated signal$$y(t)=x(t)c(t),$$

under the assumption of $\omega_1,\omega_2 < \omega_c$, the analytic signal is $$A(y(t))=x(t)e^{-j\omega_c t}.$$ Here, $x(t)$ can be extracted by $$\left|A(y(t))\right|=x(t)$$.

Now, assuming that we have two carriers, with similar assumptions ($\omega_1,\omega_2 < \omega_{c_1}, \omega_{c_2}$) $$c_2(t) = \cos(\omega_{c_1} t) + \cos(\omega_{c_2} t);\space y_2(t)=x(t)c_2(t),$$ the analytic signal is $$A(y_2(t))=x(t)\left(e^{-j\omega_{c_1} t}+e^{-j\omega_{c_2} t}\right).$$ Here, $\left|e^{-j\omega_{c_1} t} + e^{-j\omega_{c_2} t}\right|$ is not constant and therefore the demodulation is not possible.