What is energy in image processing?

There are several web articles on the topic, but, after having read them, I still don't know what "energy" is, in the context of computer vision (CV) and image processing (IP). It is related to optimisation, given that, apparently, there are the so-called "energy minimisation problems".

Why is it called "energy" (that is, why the word "energy")? I read it is because it is related to the concept of "energy" in physics, but assume that I have no idea of what energy in physics is. How would you explain what energy is in IP and CV? What is the intuition behind it?

Why do we care about energy? Given that there are the energy minimisation problems, we want to minimise energy, in such cases. I have also read that there are energy maximisation problems, so sometimes we also want to maximise energy.

I have read that it has different meanings depending on the context, but I just want to know its meaning in image processing and computer vision, that is, in the context of images.

The usage of the term "energy" in image processing context has historical reasons. It can be mapped to "brightness" or "intensity".

Imagine you have a (for matters of simplicity) greyscale image, that you want to transmit. Historically, that was done via some kind of analog modulation, where, simply speaking, dark "pixels", low intensity meant "low voltage" and bright "pixels", high intensity meant "high voltage". When looking at a CRT television, you could take the word "energy" quite literally, as an all black image will produce significantly lower energy consumption than an all white image.

The term "engergy minimisation" is usually employed, when trying to optimize some kind of algorithm or codec by comparing the difference of original signal and encoded-decoded signal. The codec would be perfect, if this difference was all zeros, so the target is "to minimise the energy" of the difference signal. Usage of "energy maximisation" is analog.

So, think of "energy" as "intensity" or "brightness" in the context of image processing.

• Can you give a concrete example of an "energy function", in the context of image processing, that needs to be minimised or maximised? – nbro Apr 8 at 13:16
• Take a bmp image. Convert it to JPEG, then back to bmp. Do the difference of the images and take a look at it. The higher you set the quality of the JPEG encoder, the nearer the difference will be to an all black image. – Max Apr 8 at 13:21

For computer vision, or any engineering problem, energy is used because it can be relatable to physical quantities.

In simple circuits, we know that power is current times the voltage: $$P(t) = I(t)V(t)\tag{1}$$ and energy is the integral of this over time: $$E = \int_{-\infty}^{+\infty} P(t) dt$$ If we assume Ohm's law $$V(t) = I(t) R$$ holds then (1) can be rewritten as $$P(t) = I^2(t)R$$ so the energy is then $$E = \int_{-\infty}^{+\infty} I^2(t) R dt$$ This form looks very much like the squared error in some optimization problems.

The nice thing about squared error problems is they often have provably optimal solutions. As a result, there are many good algorithms around for solving them.

In image processing, as in many parts of engineering, people know this background and so often apply least squares techniques to the problems that they come across: designing filters, template matching, etc.

Because they use least squares techniques, and because of the relation of those techniques to the idea of energy, they use the word energy when they talk about such problems - even if it's really just an artifact of the solution mechanism they're using.