My first question is about the quadratic functional that is used in generalized Tikhonov regularization: $$\Psi(f)=\frac{1}{2}\|f\|^2_\Gamma=f^T\Gamma f.$$ In the above equation what does $\Gamma$ represent, some special matrix?

Then, I don't understand why $\Psi(f)=\frac{1}{2}\|f'\|^2$ is preferable over $\Psi(f)=\frac{1}{2}\|f\|^2$ because it imposes a penalty on the oscillations in the solution directly, rather than just a penalty on the magnitude of the solution. Does that mean that the process is faster in the case we use derivative?

And one more question, I don't understand how to read the following figure: enter image description here

Is the part on the right a result of denoising? Under the figure is written that we the covariance $C=\Gamma^{-1}=I$$ is used.

  • $\begingroup$ Maybe I'm wrong but this seems very much like a Riemannian metric of a curved space where the Euclidean inner product is modified. Other ways to think about it probably involves a 'weighted' dot product injecting some anistropy. $\endgroup$ – Tolga Birdal Sep 18 '20 at 18:59

One way to interpret the Tikhonov Regularization is using the Maximum A Posteriori (MAP) framework.

Lets' say we have a model of the form:

$$ \boldsymbol{y} = H \boldsymbol{x} + \boldsymbol{n} $$

Where $ \boldsymbol{n} \sim N \left( 0, {\sigma}_{n}^{2} \right) $, namely Additive White Gaussian Noise, and the prior knowledge about $ \boldsymbol{x} $ is $ \boldsymbol{x} \sim N \left( 0, {\sigma}_{x}^{2} \right) $.

Then the MAP estimator is given by:

$$ \arg \min_{\boldsymbol{x}} \frac{1}{2 {\sigma}_{n}^{2}} {\left\| H \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} + \frac{1}{2 {\sigma}_{x}^{2}} {\left\| x \right\|}_{2}^{2} $$

So basically Tikhonov Regularization arises when we have prior knowledge or can model the parameters to have some kind of Gaussian Distribution.

Sometimes, our model is about the derivative (Very popular in Imaging). So we can say the derivative is distributed by Gaussian Distribution. Another way is having different covariance matrix (As in your example).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.