# What does the “smoothing optical flow” mean?

I have this algorithm for Correlation Based Pyramids of images:

• Create two Gaussian pyramids from the two input images

• Compute optical flow using “5×5” regions on smallest pyramid level.

• Smooth optical flow, and use as initial guess for higher resolution.

• Continue with next level. Search close to guess from higher resolution.

What is the third step asking me to do?

• I guess that will be boundary conditions. – Luis Andrés García Mar 13 '12 at 12:29
• Is this homework? If so, ask your teacher for clarification. – Geerten Mar 13 '12 at 13:27
• I agree that question is not written properly vis-a-vis our expectation, however, the subject is not really out of scope. Can we keep the question open? – Dipan Mehta Mar 14 '12 at 11:44

Optical ﬂow is an approximation of the local image motion based upon local derivatives in a given sequence of images. That is, it speciﬁes how much each image pixel moves between adjacent images. (The same concept extends on 3D as well).

The concept of Optical flow derives from the fact that the moving patterns cause temporal varieties of the image brightness. It is assumed that all temporal intensity changes are due to motion only. This is usually true but there are many exceptions (see below). These variations are measured using derivatives. The optical flow methods try to calculate the motion between two image frames which are taken at times $t$ and $\delta t$ at every pixel position. The following constraint equation can be used to derive the optical flow - which is called as Image constraint equation or optical flow equation:

$${{\partial I} \over {\partial x} }\Delta x + {{\partial I} \over {\partial y}} \Delta y + {{ \partial I} \over {\partial t}} \Delta t = 0$$

Refer to wiki page, or this lecture note for more details.

Thus the computation of diﬀerential optical ﬂow is, essentially, a two-step procedure:

1. measure the spatio-temporal intensity derivatives (which is equivalent to measuring the velocities normal to the local intensity structures) and

2. integrate normal velocities into full velocities, for example, either locally via a least squares calculation or globally via a regularization.

However, the above equation alone is not sufficient for computation since it involves two variables with only one equation/constraint. BKP Horn, introduced another constraint in their seminal paper.

BKP Horn, Brian Schunck "Determining Optical Flow" Artificial Intelligence, Vol. 16, No. 1–3, August 1981, pp. 185–203.

Accordingly, in a rigid body motion (with non deformation assumption), neighboring pixels have similar velocity. Based on this they derived another constraint based on velocity:

$$\nabla ^2 u = {{\partial ^2 u} \over {\partial x^2}} + { {\partial ^2 u} \over {\partial y^2}} = 0$$

and $$\nabla ^2 v = {{\partial ^2 v} \over {\partial x^2}} + { {\partial ^2 v} \over {\partial y^2}} = 0$$ where $$u = {{\delta x} \over {\delta t}} \text{ and } v = {{\delta y} \over {\delta t}}$$

The above two equations are considered as Smoothing Constraints of optical flow. Read this: cv-online page and another reference here.

• I have a question the u and v that we get are velocities or displacement of the pixels? – Debanjan Borthakur Apr 15 '19 at 20:34

Smoothing can mean applying a median filter of some size (recommended [5X5]) on the optical flow vecotr you computed at each iteration. This smoothing suppose to improve (significantly) the accuracy. in matlab:

u = medfilt2(u,[5 5]); v = medfilt2(v,[5 5]);

where u,v are the optical flow vectors you computed. Then (u,v) are the input u,v for the next iteration.

Have in mind that at each pyramid level the size of the image is different, andn therefore the size of u,v, this means that you have to upsample the previous u,v to size size of the currend pyramid level.

Goodluck.