Iterative Lucas-Kanade algorithm

Question

How is iterative refinement is applied to the estimate obtained by Lucas-Kanade algorithm? I am familiar with the following steps:

1. Estimate velocity at each pixel using one iteration of Lucas and Kanade estimation
2. Warp one image toward the other using the estimated flow field
3. Refine estimate by repeating the process

I am not able to get the meaning of above steps, more specifically, step (2) and what "warping one image toward the other" means.

Is it related to pyramidal implementation of Lucas-Kannade method? Can somebody please provide an explanation for this step?

Answer 1

let's first explain what warp is: if you apply LK for two images and you get say u=2 and v=3 for a certain pixel, in this case applying warping of one image is to increase the x-coordinate of that pixel by 2 and increase it's y-coordinate by 3, and then make this for all other pixels in the image using the associated u and v.

In step two you need to warp one image using the u and v motion vectors that you get from the first step, if your estimate to u and v from the first step is accurate then after applying the second step the two images will coincides on each other, and this means that if you apply Lucas Kanade algorithm on these two images(step 3) you will get u=0 and v=0 because the two images coincides on each others. but in real situations you will not get accurate u and v from the first iteration. so after applying the warping there will be a smaller motion between the two images and if you apply Lucas Kanade algorithm between these two images you will get a smaller u and v and you refine the motion vectors by the following:

$u_(next) = u_(previuos) + u_(current)$

$v_(next) = v_(previuos) + v_(current)$

the algorithm terminates when $u_(current)$ and $v_(current)$ are zeros or very small.

this is iterative LK without pyramids.

EDIT :

I get the idea from the following code which implements iterative LK with pyramids explained in the following paper:

B.D. Lucas and T. Kanade, "An Iterative Image Registration technique, with an Application to Stero Vision," Int'l Joint Conference Artifical Intelligence, pp. 121-130, 1981.