I have this algorithm for Correlation Based Pyramids of images:

• Create two Gaussian pyramids from the two input images

• Compute optical flow using “5×5” regions on smallest pyramid level.

• Smooth optical flow, and use as initial guess for higher resolution.

• Continue with next level. Search close to guess from higher resolution.

What is the third step asking me to do?

  • $\begingroup$ I guess that will be boundary conditions. $\endgroup$ Mar 13, 2012 at 12:29
  • 2
    $\begingroup$ Is this homework? If so, ask your teacher for clarification. $\endgroup$
    – Geerten
    Mar 13, 2012 at 13:27
  • 1
    $\begingroup$ I agree that question is not written properly vis-a-vis our expectation, however, the subject is not really out of scope. Can we keep the question open? $\endgroup$ Mar 14, 2012 at 11:44

2 Answers 2


Optical flow is an approximation of the local image motion based upon local derivatives in a given sequence of images. That is, it specifies how much each image pixel moves between adjacent images. (The same concept extends on 3D as well).

The concept of Optical flow derives from the fact that the moving patterns cause temporal varieties of the image brightness. It is assumed that all temporal intensity changes are due to motion only. This is usually true but there are many exceptions (see below). These variations are measured using derivatives. The optical flow methods try to calculate the motion between two image frames which are taken at times $t$ and $\delta t$ at every pixel position. The following constraint equation can be used to derive the optical flow - which is called as Image constraint equation or optical flow equation:

$$ {{\partial I} \over {\partial x} }\Delta x + {{\partial I} \over {\partial y}} \Delta y + {{ \partial I} \over {\partial t}} \Delta t = 0 $$

Refer to wiki page, or this lecture note for more details.

Thus the computation of differential optical flow is, essentially, a two-step procedure:

  1. measure the spatio-temporal intensity derivatives (which is equivalent to measuring the velocities normal to the local intensity structures) and

  2. integrate normal velocities into full velocities, for example, either locally via a least squares calculation or globally via a regularization.

However, the above equation alone is not sufficient for computation since it involves two variables with only one equation/constraint. BKP Horn, introduced another constraint in their seminal paper.

BKP Horn, Brian Schunck "Determining Optical Flow" Artificial Intelligence, Vol. 16, No. 1–3, August 1981, pp. 185–203.

Accordingly, in a rigid body motion (with non deformation assumption), neighboring pixels have similar velocity. Based on this they derived another constraint based on velocity:

$$ \nabla ^2 u = {{\partial ^2 u} \over {\partial x^2}} + { {\partial ^2 u} \over {\partial y^2}} = 0 $$

and $$ \nabla ^2 v = {{\partial ^2 v} \over {\partial x^2}} + { {\partial ^2 v} \over {\partial y^2}} = 0 $$ where $$ u = {{\delta x} \over {\delta t}} \text{ and } v = {{\delta y} \over {\delta t}} $$

The above two equations are considered as Smoothing Constraints of optical flow. Read this: cv-online page and another reference here.

  • $\begingroup$ I have a question the u and v that we get are velocities or displacement of the pixels? $\endgroup$ Apr 15, 2019 at 20:34

Smoothing can mean applying a median filter of some size (recommended [5X5]) on the optical flow vecotr you computed at each iteration. This smoothing suppose to improve (significantly) the accuracy. in matlab:

u = medfilt2(u,[5 5]); v = medfilt2(v,[5 5]);

where u,v are the optical flow vectors you computed. Then (u,v) are the input u,v for the next iteration.

Have in mind that at each pyramid level the size of the image is different, andn therefore the size of u,v, this means that you have to upsample the previous u,v to size size of the currend pyramid level.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.