Continuous Wavelet Transform vs Discrete Wavelet Transform

Question

The discrete wavelet transform is applied in many areas, such as signal compression, since it is easy to compute. I notice that, However, the continuous wavelet transform (CWT) is also applied to different subjects. In my opinion, the CWT is redundant and hence difficult to compute. So what are the advantages of the continuous wavelet transform?

Answer 1

On the one hand with the DWT, only a restricted choice of wavelets is available: those that implement 2-band perfect reconstruction (Daubechies, Symmlets, Coiflets, Spline). They are non-redundant, and often orthogonal or close to orthogonal, which simplifies some computations, inversion or statistical analysis, for instance. Yet, they are not quite shift-invariant. In other words, if you shift your signal by an integer number of samples, the coefficients are not "the same" as for the original one with a shift. You can read more at What is the difference between “equivariant to translation” and “invariant to translation”.

On the other hand, CWT theoretically allows a huge quantity of admissible wavelets. In practice, they are sub-sampled and the construction can be exact, but very close to (unnoticeable with a little noise in the data). And shift-invariance can be almost satisfied.

So when there is a specific wavelet shape that you want to use, because it is physically related to your system, or you want them to have precise properties to analyze your data finely (precise timing, local regularity, matched filtering), discrete approximations of CWT is often more convenient in a first instance. Notably, for phase analysis, it is quite common to use complex CWT, which people rarely do with the DWT.

Yet, when you have achieve your goals with the CWT, and efficiency still matters, you can search for a DWT domain processing that yields similar results.

For instance in Adaptive multiple subtraction with wavelet-based complex unary Wiener filters, 2012, we wanted to perform adaptive pattern subtraction with 1D seismic data. We first tried to combines DWT and FIR adaptive filters, but we were not satisfied. Then we moved to complex CWT, and were able to compute very efficiently the matched filters in the complex domain (oddly, with one-tap or unary filters on sliding frames). After that, we studied how far we could reduce the CWT redundancy and preserve the quality. Finally, we tried to go to 2D, where the redundancy is way more problematic. So we use discrete wavelet frames, and FIR filters, but we had a lot of difficulties in obtaining much better results than with the 1D CWT version (A Primal-Dual Proximal Algorithm for Sparse Template-Based Adaptive Filtering: Application to Seismic Multiple Removal, 2014). I still hope one can succeed with critically sampled DWT...

Answer 2

The CWT & DWT implementations differ in how they discretize the scale parameter used to stretch or shrink copies of the basic wavelet.

The finer grain scale parameter in the CWT can be useful for applications that require a very high-fidelity signal analysis, for example, where localization of transients or precise characterizations of signal periodicities are critical. Of course this has a higher computational cost.

Most people choose the tool (CWT, DWT, other) based on the signal-domain, task to solve and computational budget.

Answer 3

Fundamentally: DWT is orthogonal, CWT is redundant. Former packs the most information per sample, latter spreads out its decomposition. As will be explained: CWT yields vastly superior analysis information.

CWT decomposes a signal using a redundant "dictionary" of time-frequency atoms (borrowing analogy from S. Mallat). In language, a rich dictionary contains many words, and some words will have many meanings, which allows expressing a complex idea in a short sentence. Likewise, a rich dictionary of wavelets enables superior sparsity and 'expressivity'.

Per DWT's orthogonality, it is one-to-one: $N$ input points, $N$ output points. Consequently, most coefficient manipulations will lose information irrecoverably. In contrast, CWT can have $100N$ output points and enables a wide variety of manipulations, including nonlinear.

Analysis vs Synthesis

Analysis information is the very goal of transforming data in the first place: if all we cared for was preserving the input (synthesis information), simply don't do a transform. In context, analysis information includes:

• instantaneous frequency, amplitude, and phase
• frequency and amplitude modulation rates
• coherence, stability, and robustness of representation.

Loss of synthesis information nonetheless remains a significant consideration, since if synthesis information is lost, virtually all other forms of information is lost.

For any loss in synthesis information, CWT loses much less analysis information than DWT. This means we can subsample, trim, take modulus, etc. while preserving most of representative power. For modulus in particular, it's proven to globally shift frequencies of every row toward low frequencies, and most often very low frequencies, which enables much greater subsampling factors.

---

CWT analysis power

Examples of "analysis information" being utilized:

• Instantaneous phase, frequency, amplitude: Synchrosqueezing, ridge analysis (ch 4)
• Component extraction via one-integral inversion: even after nonlinear manipulations like synchrosqueezing, the CWT can remain invertible and allow extracting independent modes of a signal.
• Time-shift invariance, sparsity, robustness to noise, stability to perturbation, higher-order modulations: scattering transform. Utilizes spatial coherence of the joint 2D representation via stacked convolutions to extract robust and sparse information while minimizing loss of synthesis information via higher-order transforms.
• Frequency transposition invariance, frequency-dependent time-shift detection: Joint Time-Frequency Scattering. Builds upon time scattering and further exploits spatial coherence of the representation to extract richer information.
• Time-shift equivariance: an unsubsampled CWT is perfectly translation equivariant: $\text{CWT}_{s, t}x(t - t_0) = \text{CWT}_{s, t - t_0}x(t)$. This is desired in general, for stability and LTI-ness of representation, but also particularly in pattern localization applications.
• Amplitude filtering
• Higher SNR via higher-order wavelets: GMWs

The DWT is capable of none of this. And these are only applications of analytic CWT; there are more for CWT with real-valued wavelets. Overall the CWT permits for a much wider, more flexible family of wavelets than DWT. What's missing from the list is compression, which DWT is also capable of, but CWT can do it even better.

The scattering transform has achieved or surpassed SOTA on many datasets with limited data (one does not simply beat DNNs on big data).

---

CWT myths

Not exhaustive list:

• Scalogram is not invertible: yes it is, with small caveats (TL;DR: weak general invertibility, strong invertibility for natural signals, perfect invertibility for unimodal signals). Note "scalogram" is modulus of CWT; raw CWT is much easier to invert perfectly.
• CWT is slower than STFT: only because most implementations don't know better. CWT can very much have a "hop size" and be implemented column-wise rather than row-wise.
• CWT is for continuous, DWT is for discrete: misguided distinction. CWT bases are compactly supported and avoid artifacts in sense of continuous FT vs DFT (though they have their own share of problems, in both domains). CWT is very much applicable to discrete signals.

---

STFT & DFT, related posts

Much of what's said in this post also applies for STFT vs DFT. Related posts:

• Advantage of overlapping STFT windows
• Advantages of complex wavelets

---

CWT to explain success of Convolutional Neural Networks

Why are CNNs built with nonlinear activations, max pooling, and multiple layers? The best explanation I know of is in terms of the CWT, presented by S. Mallat.

---

CWT is hard

This is perhaps the main disadvantage: to make best use of it, one requires a good deal of familiarity. The CWT is also very easy to implement inefficiently or incorrectly: scipy, PyWavelets, and even MATLAB are all incorrect in some way, MATLAB being the best of (example; MATLAB being the best of three). The best implementation I know if is Kymatio's (disclaimer, I'm a contributor).

The crux of the difficulty is wavelet design: I will eventually write about it in this and a standalone post. You can be notified by bookmarking or via Github.