Questions on the Generalized Tikhonov Regularization

Question

My first question is about the quadratic functional that is used in generalized Tikhonov regularization: $$\Psi(f)=\frac{1}{2}\|f\|^2_\Gamma=f^T\Gamma f.$$ In the above equation what does $\Gamma$ represent, some special matrix?

Then, I don't understand why $\Psi(f)=\frac{1}{2}\|f'\|^2$ is preferable over $\Psi(f)=\frac{1}{2}\|f\|^2$ because it imposes a penalty on the oscillations in the solution directly, rather than just a penalty on the magnitude of the solution. Does that mean that the process is faster in the case we use derivative?

And one more question, I don't understand how to read the following figure:

Is the part on the right a result of denoising? Under the figure is written that we the covariance $C=\Gamma^{-1}=I$$ is used.

Answer 1

One way to interpret the Tikhonov Regularization is using the Maximum A Posteriori (MAP) framework.

Lets' say we have a model of the form:

$$ \boldsymbol{y} = H \boldsymbol{x} + \boldsymbol{n} $$

Where $ \boldsymbol{n} \sim N \left( 0, {\sigma}_{n}^{2} \right) $, namely Additive White Gaussian Noise, and the prior knowledge about $ \boldsymbol{x} $ is $ \boldsymbol{x} \sim N \left( 0, {\sigma}_{x}^{2} \right) $.

Then the MAP estimator is given by:

$$ \arg \min_{\boldsymbol{x}} \frac{1}{2 {\sigma}_{n}^{2}} {\left\| H \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} + \frac{1}{2 {\sigma}_{x}^{2}} {\left\| x \right\|}_{2}^{2} $$

So basically Tikhonov Regularization arises when we have prior knowledge or can model the parameters to have some kind of Gaussian Distribution.

Sometimes, our model is about the derivative (Very popular in Imaging). So we can say the derivative is distributed by Gaussian Distribution. Another way is having different covariance matrix (As in your example).