When I read papers on image denoising, I always encounter sparse representation. For image denoising, we try to separate image signal from noise. It is assumed that signal is correlated and noise is uncorrelated. Sparse representation represents one signal as a linear combination of a small number of dictionary elements. It hope to use as few as non-zero coefficients to represent signal. In my view, it is appropriate for image compression. How can it is related to image denoising?
I will start the explanation from the compression viewpoint. There are two main types: lossless compression, and lossy compression. Noise, at least divergence or loss from the original data, arises only with lossy compression.
When one considers the original data as the "clean" reference, lossy compression adds a amount of loss related (generally vaguely increasing) to the compression ratio allowed.
However, it is interesting to take another perspective, which I borrow from Filtering Random Noise from Deterministic Signals via Data Compression, 1995, B. K. Natarajan, that I found influential.
The main idea starts from the observation that noise (with a sense of randomness, lack of predictability) is hard to compress, while structured signals possess a certain amount of correlation that somehow can be compacted. In other words, when data is easy to compress, it is likely to be structured.
This led Natarajan to the definition of Occam filters, a name related to W. of Ockham and his (alleged) law of parsimony. Starting from noisy data, one compresses it to a loss close to the original noise power. If the compression system is well designed, this tends to remove (partly) some noise from the original signal. Shocking enough, lossy compression can denoise, killing two birds (denoising and compression) with one stone. This can be found for instance in Adaptive wavelet thresholding for image denoising and compression, 2000, where the noise limit can be made adaptive.
Sparse representations, such as the wavelets above, but other transforms as well, have been found since effective at concentrating the energy of the structured signals, and at spreading or whitening some random noises. Being sparse, the sparse representation of the image has structures easy to compress, and spread noises. The very same action of obtaining a sparse representation is a good preprocessing for both compression and denoising.
Note however that some abuse about "compression". Actual compression encompasses quantization, coefficient location, entropy coding, etc., and merely producing few large coefficients and many zeroes does not really qualify for compression.
Finally, the analogy between compression and image denoising is not valid for all images, all types and levels of noises, but it is sufficiently correct in standard cases to allow them to be taught together.
As for comments:
if you have a sparse or compressible signal $x$ (in some representation $f$), and add uncorrelated noise $n$ to it, then applying $f$ on $x+n$ is not denoising per se, because for denoising you need to decide, or choose, what to keep as a signal component and what to filter as a noise. But it generally provides:
- a clearer "distinction" between them, in terms of (local) amplitude, that often simplify the use of statistical tools, or make them more robust
- better access to signal, or noise, properties, to parametrize the denoising
orthogonal transformations are quite good, but the orthogonality limit their ability to sparsely represent signals. Overcomplete transforms induce correlations in the noise (Noise Covariance Properties in Dual-Tree Wavelet Decompositions), but as the signal coefficients are sparser, the relative amplitude of (sparse) signal coefficients and noise is (optimally) more favorable. Examples of correlated noise whitening with for instance redundant complex wavelets can be found for instance in the above reference or Removal of Correlated Noise by Modeling the Signal of Interest in the Wavelet Domain.
Of course, a lot the above assumes that the signal is sparse, that the noise is not correlated, and that you are able to find a proper representation, which is not evident in practice.
Here are two interpretations of why sparse representations are useful for denoising:
1) Your clean image is well represented by your dictionary (usually a wavelet or 2D-DCT dictionary), but the noise isn't. This means that when calculating your sparse representation from the noisy signal, you will only capture the clean signal, and not the noise.
2) Enforcing the representation vector to be sparse (i.e. with only a few non-zero elements), means that you will eventually threshold out all the other low-energy noisy components.
As you said sparse representations can be pretty useful for compression. Compression is merely about keeping the relevant information, while discarding the irrelevant one. Well denoising is about keeping the relevant part of your signal (the one that fits your model, or your dictionary), while discarding the irrelevant one (the noise).