[EDIT: some code made available] A common framework for (multivariate) image processing is to suppose that its useful features (edges, textures, spectral correlation) contain redundancy, while the noise is pretty uncorrelated. A major preprocessing task consists in decorrelating or compacting the image features prior to processing, hoping to separate them from unwanted noises better in some transformed domain, or by projection onto a set of vectors $v_i$.
One holy grail in image $I$ decomposition is to find the best $M$-term approximation, so that a norm $\|I-\sum_1^M \lambda_i v_i\|$ is minimal, when $M$ is small with respect to the size of the image. Such a problem is generally very difficult. So many methods try to find the best $v_i$s or $\lambda_i$ (or both) under some assumptions, or using optimization methods.
Two main types of linear transformations are usually considered:
- adaptive transformations, dependent on image characteristics
- fixed transformations, independent of images, but generally designed to capture features of image classes or models
SVD, PCA, Karhunen-Loève transformations belong to Type 1. Fourier, DCT and wavelets belong to Type 2. However, they are somehow related, and under some assumptions, Fourier derives from PCA, and DCT is a good approximation to Karhunen-Loève. They are all related to minimum squared norm formulations, provide "orthogonal" vectors $v_i$, and very often, one considers that the energy of the projection coefficients $\lambda_i$ onto the $v_i$s are good enough representations for practical purposes, less efficient than best $M$-terms, but easier to compute.
Now, the bigger the $\lambda_i^2$, the more it contributes to the energy of the image. Resultingly, as long as the vectors have the same norm, it is a good bet to select the most important projection coefficients $|\lambda_i$| (in magnitude), to provide a sparse representation of the image.
Then, depending on the direction (spatial, spectral), fixed and adapted transformations can be combined, such as PCA (or other learnt linear transformations) and wavelets. Three additional sources on multivariate denoising for you: