Position vector of Lukas-Kanade optical flow algorithm

Question

The classical optical flow equation by Lukas-Kanade can be simplified as follow: $$ \frac{\delta I}{\delta x}\triangle x+\frac{\delta I}{\delta y}\triangle y + \frac{\delta I}{\delta t}\triangle t=0 $$

From here I can approximate the position vector (to the direction of intensity gradient) by: $$ \triangle s=\sqrt{\left(\frac{\frac{\delta I_{t+1}}{\delta x}}{\lvert\triangle I_t\rvert}\right)^2+{\left(\frac{\frac{\delta I_{t+1}}{\delta y}}{\lvert \triangle I_t\rvert}\right)^2}} $$

Where

$$ \lvert \triangle I_t\rvert=\sqrt{\left(\frac{\delta I_{t}}{\delta x}\right)^2+\left(\frac{\delta I_{t}}{\delta y}\right)^2} $$

However, when I try it out in MATLAB it seems that the displacement is not correct. See the code below, where I and I2 are the intensity image $I_t$ and $I_{t+1}$ respectively.

%generate two images
I=[255  255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 0   255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255]; 

I2=[255 255 255 255 255 255 255 255 255 255
255 255 255 255 0   255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255 255 255];

I=im2double(I);
I2=im2double(I2);
[Ix, Iy]=imgradientxy(I,'sobel');
[Ix2, Iy2]=imgradientxy(I2,'sobel');
I_scalar=sqrt(Ix.^2+Iy.^2);
intshift=sqrt(round(Ix2./I_scalar).^2+round(Iy2./I_scalar).^2);

As you can see, it has shifted $x=2$, $y=6$ (for the 0 value intensity). Which the intshift should be $6.32$. However that's not the case.

Would anyone please tell me where I did wrong?

Answer 1

Lukas-Kanade uses a local window to determine displacement. The Sobel gradient operator uses a 3x3 window. If something moves more than one pixel distance, this 3x3 window can no longer see the shifted point from its original location, and thus the given equations cannot relate these two points to each other.

Applying a sufficiently large Gaussian window before computing the derivatives (or better yet, use a Gaussian derivative operator instead of the Sobel operator) will allow the two points to be matched, but will also reduce the precision of the match.

The typical solution is to apply Lukas-Kanade in a multi-scale manner, where one first estimates shifts at a large scale, transforms one of the images accordingly, and repeats with decreasing scale until sufficient precision is obtained. All the estimated displacements must then be added together to obtain the actual displacement.