What exactly is meant by "translation invariant dictionaries/wavelets"?

I'm trying to wrap my head around the notion of translation invariance in terms of dictionaries/wavelets. For example in Lecture Notes, Page 41 its written that one starts with a family of atoms/wavelets $$(\psi_j)_{j\in \mathbb{Z}}$$ and adds all translates $$t\in \mathbb{R}$$ via defining $$\begin{equation*} \psi_{j,t}(x) := \psi_j (x - t). \end{equation*}$$ Now the calculation of the coefficients for a singal $$f$$ becomes $$\begin{equation}\label{calc coeff} \Phi f (j,t) = \left\langle f, \psi_{j,t}\right\rangle = \int_\mathbb{R} f(x) \psi_j^\ast (t - x) \mathrm{d}x = f \ast \psi_j^\ast (t), \end{equation}$$ where $$\psi_j^\ast(x) := \psi_j(-x)$$. To me, translation invariance means something like $$f(x) = f(x-t)$$, but I don't understand/see where this equations holds for the above "argumentation"?

Furthermore, reading some Wikipedia on Stationary Wavelet Transform, it says in the first sentence "to overcome the lack of translation-invariance of the discrete wavelet transform", i.e. when the translation $$t\in \mathbb{R}$$ becomes $$t\in \mathbb{Z}$$. Why is the discrete wavelet transform not translation invariant? The calculation for the coefficients $$\left\langle f, \psi_{j,t} \right\rangle$$ is the same, but only discrete convolution instead of continuous?

It's rather translation equivariant:

$$\text{CWT}_{s, t}x(t - t_0) = \text{CWT}_{s, t - t_0}x(t) \tag{1}$$

and

$$\langle x(t−t_0),\psi(t) \rangle= \langle x(t), \psi(t+t_0)\rangle \tag{2}$$

That is, when a signal is shifted, its representation is also shifted but not modified (like LTI). This makes its derived features, such as energy and norm, or most manipulations of coefficients, invariant - hence the terminology (though agreeably misleading).

DWT

The DWT is likewise equivariant at source - i.e. before subsampling. Suppose subsampling by x2:

$$[0, 1, 2, 3, 4, 5, 6, 0, 0] \rightarrow [0, 2, 4, 6, 0]$$ now (circular-)shift by 1: $$[0, 0, 1, 2, 3, 4, 5, 6, 0] \rightarrow [0, 1, 3, 5, 0]$$

If it were equivariant, we'd get $$[0, 0, 2, 4, 6]$$ - but now, our manipulations (e.g. $$\sum |\text{coef}|^2$$) will no longer yield the same results. Nonetheless, "at source" equivariance is much more than no equivariance at all, and some manipulations can take advantage of it.

Equivariant dictionaries

One can notice that $$(2)$$ is an identity - i.e. it's always true. The idea is that "shift" for wavelets is defined this way to begin with, i.e. "shift by $$t_0$$" means $$\psi(t - t_0)$$, which isn't the case for all functions (e.g. its Fourier transform, shifted as a function of frequency, also changes its width, breaking equivariance). To define a shift this way means to make it a convolutional operator, and the derived representation LTI.

Invariance

There is an important sense in which complex wavelets are invariant:

$$\text{CWT}(x(t)) = \text{CWT}(x(t - t_0))$$

While the equality never strictly holds with wavelets alone (except at infinite scale), the distance between shifted coefficients can be made as low as the application demands with further steps; with wavelets alone, the greater the scale, the lesser the distance. Time-shift invariance is a bedrock of the scattering transform.

See related post.

but only discrete convolution instead of continuous

The distinction is unimportant: they're both implemented as discrete convolutions. The difference is in subsampling scheme and choice of wavelets.

• Thanks that clarified already a lot! I would have a follow-up question: For the last point on distinction between discrete/continuous, could you give examples of wavelets which can be used both, for CWT and DWT and/or which can be used for one but not the other? (I'm mostly familiar the analytical definition of Shannon-, Haar- or Meyer-Wavelet) Aug 20 '21 at 10:51
• @stish Any DWT wavelet can be used for CWT, but not other way around; most CWT wavelets are ill-suited (or invalid) for DWT. E.g. Morlet, Morse, Mexican Hat, Bump - anything that cannot effectively tile the frequency domain without overlaps. Aug 20 '21 at 11:16