I have this algorithm for Correlation Based Pyramids of images:
• Create two Gaussian pyramids from the two input images
• Compute optical flow using “5×5” regions on smallest pyramid level.
• Smooth optical flow, and use as initial guess for higher resolution.
• Continue with next level. Search close to guess from higher resolution.
What is the third step asking me to do?
Optical flow is an approximation of the local image motion based upon local derivatives in a given sequence of images. That is, it specifies how much each image pixel moves between adjacent images. (The same concept extends on 3D as well).
The concept of Optical flow derives from the fact that the moving patterns cause temporal varieties of the image brightness. It is assumed that all temporal intensity changes are due to motion only. This is usually true but there are many exceptions (see below). These variations are measured using derivatives. The optical flow methods try to calculate the motion between two image frames which are taken at times $t$ and $\delta t$ at every pixel position. The following constraint equation can be used to derive the optical flow - which is called as Image constraint equation or optical flow equation:
$$ {{\partial I} \over {\partial x} }\Delta x + {{\partial I} \over {\partial y}} \Delta y + {{ \partial I} \over {\partial t}} \Delta t = 0 $$
Refer to wiki page, or this lecture note for more details.
Thus the computation of differential optical flow is, essentially, a two-step procedure:
measure the spatio-temporal intensity derivatives (which is equivalent to measuring the velocities normal to the local intensity structures) and
integrate normal velocities into full velocities, for example, either locally via a least squares calculation or globally via a regularization.
However, the above equation alone is not sufficient for computation since it involves two variables with only one equation/constraint. BKP Horn, introduced another constraint in their seminal paper.
Accordingly, in a rigid body motion (with non deformation assumption), neighboring pixels have similar velocity. Based on this they derived another constraint based on velocity:
$$ \nabla ^2 u = {{\partial ^2 u} \over {\partial x^2}} + { {\partial ^2 u} \over {\partial y^2}} = 0 $$
and $$ \nabla ^2 v = {{\partial ^2 v} \over {\partial x^2}} + { {\partial ^2 v} \over {\partial y^2}} = 0 $$ where $$ u = {{\delta x} \over {\delta t}} \text{ and } v = {{\delta y} \over {\delta t}} $$
The above two equations are considered as Smoothing Constraints of optical flow. Read this: cv-online page and another reference here.
Smoothing can mean applying a median filter of some size (recommended [5X5]) on the optical flow vecotr you computed at each iteration. This smoothing suppose to improve (significantly) the accuracy. in matlab:
u = medfilt2(u,[5 5]); v = medfilt2(v,[5 5]);
where u,v are the optical flow vectors you computed. Then (u,v) are the input u,v for the next iteration.
Have in mind that at each pyramid level the size of the image is different, andn therefore the size of u,v, this means that you have to upsample the previous u,v to size size of the currend pyramid level.
Goodluck.
