12
$\begingroup$

I'm a masters student, preparing a seminar in computer vision. Among the topics is the Kanade-Lucas-Tomasi (KLT) tracker, as described in

J. Shi, C. Tomasi, "Good features to track". Proceedings CVPR '94.

Here's a web resource that I'm using to understand the KLT tracker. I need some help with the math, as I'm a bit rusty in linear algebra and have no prior experience with computer vision.

In this formula for $\Delta p$ (step 5 in the summary), note the inverse Hessian:

$$\Delta p = H^{-1}\Sigma_x\left[\nabla I \frac{\partial W}{\partial p}\right]^\mathsf{T} \left[T(x) − I(W(x; p))\right]$$

In the article, good features to track are defined as ones where the sum of inverse Hessian matrices have large, similar eigenvalues: $\min(\lambda_1,\lambda_2)>threshold$. I was unable to understand how and where this is derived from, mathematically.

The intuition is that this represents a corner; T get that. What does that have to do with eigenvalues? I expect that if the values of the Hessian are low, there's no change, and it's not a corner. If they're high, it's a corner. Does anyone know how the intuition of cornerness comes into play in the eigenvalues of the inverse Hessian in order to determine $\Delta p$ across iterations of the KLT tracker?

I've been able to find resources claiming that the inverse Hessian correlates to the image covariance matrix. Furthermore, the image covariance indicates the intensity change, and then it makes sense... but I've been unable to find what exactly an image covariance matrix is with respect to an image, and not a vector, or a collection of images.

Also, eigenvalues have meaning in principle component analysis, which is why I get the idea for an image covariance matrix, but I'm not sure how to apply this to the Hessian, as it's usually applied to an image. The Hessian, as far as I understand, is a $2\times 2$ matrix defining the 2nd derivatives for $x$, $y$, and $xy$ at a certain location $(x,y)$.

I would really appreciate help with this, as I've been on it for 3+ days, it's just one small formula and time is running out.

Improve this question
$\endgroup$
1
  • $\begingroup$ ok, i've pretty much got this through a bunch of web-resources concerning principal curvature, differential geomatry, matrix condition number (well conditioned matrix). i still need to formulate a reasonable explanation for the seminar. once i have it i will either publish it here, or link this page to the seminar. $\endgroup$
    – nathan g
    Commented Oct 25, 2011 at 23:45

1 Answer 1

Reset to default
5
$\begingroup$

Think of them as 2D smoothness terms.
The smoother the patch, the lower the matrix rank and the closer the matrix is to being singular.

On an straight edge (not a corner), just one eigenvalue will be large.
On a corner both will be large.

Using eigenvalues means that the angle of the edge is not a factor, and at any angle, an edge will give just one large e.v.

Improve this answer
$\endgroup$
3
  • $\begingroup$ thank you for your answer. i've found many resources giving similar intuitions, and discussing the aperture problem. the intuition is and was clear. my question was more mathematical in nature, and once i found the answer it turns out it was much simpler. just basic matrix properties. similar eigenvalues mean the matrix is well conditioned, and the max eigenvalue is bounded, so giving a lower bound makes the eigenvalues similar. further more, the eigenvalues correlate to principal curvatures, for the hessian. this is the information i was looking for at the time. $\endgroup$
    – nathan g
    Commented Nov 11, 2011 at 23:52
  • $\begingroup$ i re-read your answer, and i find the comment concerning the eigenvalues and the angle insightful. thank you for sharing that with me. $\endgroup$
    – nathan g
    Commented Nov 11, 2011 at 23:57
  • $\begingroup$ You should mark it as "Answered" then. $\endgroup$
    – Adi Shavit
    Commented Jan 4, 2018 at 12:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.