Optical flow equation and real images

Question

We know optical flow equation cannot be calculated without additional constraints. But what I cannot figure out is why it does not hold exactly for real images? This equation has gradient and so, it can be used to estimate the motion vectors by focusing on edges. Is it correct? How we add other constraints like smoothness into account? Thanks

Answer 1

Writing the optical flow vector as $(u,v)$, then the optical flow equation only states that light is preserved. Basically, it states that variations in the observed light patterns (measured by the image derivative with respect to time) are explained by the motion of objects (smaller light patterns inside the image). Object motion is measured through spatial variations of the said light patterns:

$$\frac{\partial I}{\partial t} + u\cdot\frac{\partial I}{\partial x} + v\cdot\frac{\partial I}{\partial y} = 0$$

This problem is ill-posed for any image, be it synthetic or real. This is easy to see since 1 pixel provides 1 measurement $\frac{\partial I}{\partial t}$ only, while at the same time there are two unknowns $\left(u(x,y), v(x,y)\right)$.

As you said, since this equation involves image spatial gradients $\nabla I = (\frac{\partial I}{\partial x}, \frac{\partial I}{\partial y})$ it can be solved for edges (by tracking them). Other ways to overcome this issue:

• assuming that the optical flow is constant over a square of several pixels and then solving the resulting system. Typically, using 3x3 squares yields 9 equations and a least squares solution to the optical flow can be computed;
• more involved approaches rely on regularized problems. Regularized approaches look for a flow that will fit the observed data while having good properties such as smoothness, exhibiting thin discontinuities...