Justification for Squared $ {L}_{2} $ Data and Smoothness Term as an Error Bound

Question

Often in variational methods (and not only) we have an energy that is of the form: $$E(u) = \frac{1}{2}\|f-u\|^2_2 + \frac{\alpha}{2}\|\psi(u)\|^2_2,$$

where the first term is referred to as the data term, and the second as the smoothness term. I understand that the squared $L_2$ norm is especially appealing since it results in a simple expression when we are looking for the minimum. I am looking for the formal motivation for designing such an energy. More specifically, I would like to see it as resulting from some bound upon the actual error. Let $f$ be a degraded/noisy image of the original image $g$, and $u$ be the image that we are looking for. Is it possible for the above to be interpreted as a bound of $\|g-f\|^2_2$? Let us consider an even simpler problem (where $h$ is a sufficiently smoothed version of $f$): $$E(u) = \frac{1}{2}\|f-u\|^2_2 + \frac{\alpha}{2}\|u-h\|^2_2$$ If I start from $\|g-f\|_2$ I can get the following bound: $$\|u-g\| = \|u-f+f-g\| \leq \|u-f\| + \|f-g\|$$ $$\|u-g\| = \|u-h+h-g\| \leq \|u-h\| + \|h-g\|$$ Combining the two inequalities with some weight $\lambda \in [0,1]$ I get: $$\|u-g\|\leq (1-\lambda)\|u-f\| + \lambda\|u-h\| + (1-\lambda)\|f-g\| + \lambda\|h-g\|$$ Since $f,g,h$ are fixed, the minimization is only over the first two terms: $$\min_u (1-\lambda)\|u-f\| + \lambda\|u-h\|$$ And this can be rewritten through $\alpha$ by setting $\alpha = \frac{\lambda}{1-\lambda}$. My issue is that these are still not squared norms. If I try to square both sides, this results in a different minimization energy. The other way to get the squares involves writing out something of the form: $$\|u-g\|^2 = \|(u-f) + (f-g)\|^2 = \|u-f\|^2 + \|f-g\|^2 - 2(u-f)\cdot(g-f) \\ \leq \|u-f\|^2 + \|f-g\|^2$$ The last inequality holds only when $(u-f)\cdot(g-f) \geq 0$ though. Said otherwise, the angle between $(u-f)$ and $(g-f)$ needs to be smaller than 90 degrees which is not something that can be guaranteed I believe. A looser bound is: $$\|u-g\|^2 \leq \|u-f\|^2 + \|f-g\|^2 + 2\|u-f\|\|f-g\|,$$ but it also involves non-squared terms.

Here's something else I noticed about the non-squared energy:

$$\alpha\|u-f\| + \beta\|u-h\|, \alpha, \beta > 0$$

will always have as solutions $u=f$ or $u=h$ for $\alpha<\beta$ respectively $\alpha>\beta$. When $\alpha=\beta$ any solution of the form $(1-\lambda)f + \lambda h$ is admissible. It is thus clear that the above is not very interesting as a minimization energy.

Cross-posted to: https://math.stackexchange.com/q/3692290/463794 I expect to get a different perspective there.

Answer 1

One of the motivations to use the $ {L}_{2} $ norm comes from the Maximum a Posteriori Estimation (MAP) framework.

If you model $ \psi \left( u \right) \sim \mathcal{N} \left( 0, \alpha \right) $ then if you derive the MAP Estimator in case the added noise is Gaussian you'd get the exact model you posted above.

An example of the derivation of MAP model to the above can be seen in my answer to Estimating the Signal by Deconvolution with a Prior on the Filter Coefficients and the Signal Samples.

Answer 2

I figured out how to show that after some time. It's just Jensen's inequality wrt $\|\cdot\|^2_2$ which is a convex function. That is, I first apply the triangle inequality to:

$$\|u-g\|_2 = \|(1-\gamma)(u-f)+\gamma(u-h) + (1-\gamma)(f-g) + \gamma(h-g)\|_2 \leq \|(1-\gamma)(u-f)+\gamma(u-h)\|_2 + \|(1-\gamma)(f-g) + \gamma(h-g)\|_2.$$

Since the second term is irrelevant wrt $u$ we can focus on the first term:

$$\|(1-\gamma)(u-f)+\gamma(u-h)\|^2_2 \leq (1-\gamma)\|u-f\|^2_2 + \gamma\|u-h\|^2_2.$$

Then the minimization problem reads:

$$\arg\min_{u\in\Omega}\,(1-\gamma)\|u-f\|^2_2 + \gamma\|u-h\|^2_2$$