What exactly is meant by "translation invariant dictionaries/wavelets"?

Question

I'm trying to wrap my head around the notion of translation invariance in terms of dictionaries/wavelets. For example in Lecture Notes, Page 41 its written that one starts with a family of atoms/wavelets $(\psi_j)_{j\in \mathbb{Z}}$ and adds all translates $t\in \mathbb{R}$ via defining \begin{equation*} \psi_{j,t}(x) := \psi_j (x - t). \end{equation*} Now the calculation of the coefficients for a singal $f$ becomes \begin{equation}\label{calc coeff} \Phi f (j,t) = \left\langle f, \psi_{j,t}\right\rangle = \int_\mathbb{R} f(x) \psi_j^\ast (t - x) \mathrm{d}x = f \ast \psi_j^\ast (t), \end{equation} where $\psi_j^\ast(x) := \psi_j(-x)$. To me, translation invariance means something like $f(x) = f(x-t)$, but I don't understand/see where this equations holds for the above "argumentation"?

Furthermore, reading some Wikipedia on Stationary Wavelet Transform, it says in the first sentence "to overcome the lack of translation-invariance of the discrete wavelet transform", i.e. when the translation $t\in \mathbb{R}$ becomes $t\in \mathbb{Z}$. Why is the discrete wavelet transform not translation invariant? The calculation for the coefficients $\left\langle f, \psi_{j,t} \right\rangle$ is the same, but only discrete convolution instead of continuous?

Answer 1

It's rather translation equivariant:

$$ \text{CWT}_{s, t}x(t - t_0) = \text{CWT}_{s, t - t_0}x(t) \tag{1} $$

and

$$ \langle x(t−t_0),\psi(t) \rangle= \langle x(t), \psi(t+t_0)\rangle \tag{2} $$

That is, when a signal is shifted, its representation is also shifted but not modified (like LTI). This makes its derived features, such as energy and norm, or most manipulations of coefficients, invariant - hence the terminology (though agreeably misleading).

DWT

The DWT is likewise equivariant at source - i.e. before subsampling. Suppose subsampling by x2:

$$ [0, 1, 2, 3, 4, 5, 6, 0, 0] \rightarrow [0, 2, 4, 6, 0] $$ now (circular-)shift by 1: $$ [0, 0, 1, 2, 3, 4, 5, 6, 0] \rightarrow [0, 1, 3, 5, 0] $$

If it were equivariant, we'd get $[0, 0, 2, 4, 6]$ - but now, our manipulations (e.g. $\sum |\text{coef}|^2$) will no longer yield the same results. Nonetheless, "at source" equivariance is much more than no equivariance at all, and some manipulations can take advantage of it.

Equivariant dictionaries

One can notice that $(2)$ is an identity - i.e. it's always true. The idea is that "shift" for wavelets is defined this way to begin with, i.e. "shift by $t_0$" means $\psi(t - t_0)$, which isn't the case for all functions (e.g. its Fourier transform, shifted as a function of frequency, also changes its width, breaking equivariance). To define a shift this way means to make it a convolutional operator, and the derived representation LTI.

Invariance

There is an important sense in which complex wavelets are invariant:

$$ \text{CWT}(x(t)) = \text{CWT}(x(t - t_0)) $$

While the equality never strictly holds with wavelets alone (except at infinite scale), the distance between shifted coefficients can be made as low as the application demands with further steps; with wavelets alone, the greater the scale, the lesser the distance. Time-shift invariance is a bedrock of the scattering transform.

See related post.

---

but only discrete convolution instead of continuous

The distinction is unimportant: they're both implemented as discrete convolutions. The difference is in subsampling scheme and choice of wavelets.