What is energy in image processing?

Question

There are several web articles on the topic, but, after having read them, I still don't know what "energy" is, in the context of computer vision (CV) and image processing (IP). It is related to optimisation, given that, apparently, there are the so-called "energy minimisation problems".

Why is it called "energy" (that is, why the word "energy")? I read it is because it is related to the concept of "energy" in physics, but assume that I have no idea of what energy in physics is. How would you explain what energy is in IP and CV? What is the intuition behind it?

Why do we care about energy? Given that there are the energy minimisation problems, we want to minimise energy, in such cases. I have also read that there are energy maximisation problems, so sometimes we also want to maximise energy.

I have read that it has different meanings depending on the context, but I just want to know its meaning in image processing and computer vision, that is, in the context of images.

Answer 1

The usage of the term "energy" in image processing context has historical reasons. It can be mapped to "brightness" or "intensity".

Imagine you have a (for matters of simplicity) greyscale image, that you want to transmit. Historically, that was done via some kind of analog modulation, where, simply speaking, dark "pixels", low intensity meant "low voltage" and bright "pixels", high intensity meant "high voltage". When looking at a CRT television, you could take the word "energy" quite literally, as an all black image will produce significantly lower energy consumption than an all white image.

The term "engergy minimisation" is usually employed, when trying to optimize some kind of algorithm or codec by comparing the difference of original signal and encoded-decoded signal. The codec would be perfect, if this difference was all zeros, so the target is "to minimise the energy" of the difference signal. Usage of "energy maximisation" is analog.

So, think of "energy" as "intensity" or "brightness" in the context of image processing.

Answer 2

For computer vision, or any engineering problem, energy is used because it can be relatable to physical quantities.

In simple circuits, we know that power is current times the voltage: $$P(t) = I(t)V(t)\tag{1}$$ and energy is the integral of this over time: $$E = \int_{-\infty}^{+\infty} P(t) dt $$ If we assume Ohm's law $$ V(t) = I(t) R$$ holds then (1) can be rewritten as $$ P(t) = I^2(t)R$$ so the energy is then $$E = \int_{-\infty}^{+\infty} I^2(t) R dt $$ This form looks very much like the squared error in some optimization problems.

The nice thing about squared error problems is they often have provably optimal solutions. As a result, there are many good algorithms around for solving them.

In image processing, as in many parts of engineering, people know this background and so often apply least squares techniques to the problems that they come across: designing filters, template matching, etc.

Because they use least squares techniques, and because of the relation of those techniques to the idea of energy, they use the word energy when they talk about such problems - even if it's really just an artifact of the solution mechanism they're using.

Answer 3

Energy is a useful, and powerful, proxy In many data restoration tasks, when one is uncertain about the data, it is tempting to average them to reduce noise. What most people have forgotten is that "averaging values", ie summing values and dividing the sum by the cardinal, is an algorithm, not a definition. A proper optimization-like definition to the average is:

the quantity that minimizes the squared distance to observed values.

In other words, the average is:

$$ \hat{m} =\arg \min \sum_k (x_k-m)^2$$

If you add weights ($\sum_k w_k(x_k-m)^2$), you recover all linear filters, that bear a close relationship to Gaussian statistics, and of course orthogonality. Minimizing an energy is relatively tractable, as the differentiation of a square may turn into linear equations.

In image processing, one feels that sum approximations (with parameters $\theta$) $A_\theta(I)$ to an image $I$ can be appropriate: approximating the image by its mere averages, or by a limited sum of weighted functions. This can be cast into an energy minimization problem:

$$ \hat{I} =\arg \min (I-A_\theta(I))^2$$

$A_\theta(I)$ can be a degradation operator on the image: blurring, saturation, etc. As this cost function is minimized by the image itself, it is not sufficient to account for image degradation: noise, smearing, blur, motion, etc. Thus, many works use a least energy fidelity term, and often add a penalty $P_\tau$ that accounts for image specificity: total variation, sparsity, positivity, etc. as:

$$ \hat{I} =\arg \min (I-A_\theta(I))^2+ P_\tau(A)$$