# SURF feature extractor and descriptor

I am trying to understand how SURF extracts features using a Hessian matrix. What I am a bit confused about, is why a second derivative Gaussian filter is used, rather than a standard Gaussian filter? What effect does this have on the system?

First of all, SURF only uses it as a blob detector to find interest points. The first order derivatives only give you directional information such as edge-ness. However, the Hessian (as the matrix of second derivatives) contains local structure information such as curvature / concavity etc. For instance, its eigenvectors can be used for cornerness measures. It is just more informative. Also, its determinant is used as an automatic scale selector.

Another common alternative would be a Laplacian detector, but Hessian shows better scale selection properties.

• maybe we could obtain corner information from Hessian, but originally we use second order moment or structure tensor for that purpose which is different from Hessian. – Mohammad M Jun 6 '17 at 7:35
• Methods like SIFT use a second order moment matrix, but not SURF. SURF does use the Hessian. – Tolga Birdal Jun 6 '17 at 8:25
• i was trying to say we use second moment for corners and hessian used for bloblike features. in other words hessian doesn't give us corner information at least directly (you said by its eigenvectors) – Mohammad M Jun 6 '17 at 9:23
• Also the original SIFT use DoG which is an approximation of scaled Laplacian and it doesn't use second moment. Second moment used by Harris detectors. – Mohammad M Jun 6 '17 at 9:29
• Ok agreed then. – Tolga Birdal Jun 6 '17 at 9:44

SURF detector is kind of blob detector which find the local maxima of DoH (determinant of Hessian) over scale space to find both coordinate and the scale of feature point.

Hessian is the matrix of second order derivatives. To estimate Hessian at different scales we have to perform convolution with Gaussian to obtain image at different scales also take the derivative by finite difference approximations. Now considering convolutions and taking derivatives are linear operator we could interchange their order or combine them, so instead of performing these operations seperately we perform the combination of these two operation by applying the derivatives o Gaussian as convolution kernel.

By the way in SURF method to speed up calculations we don't use Gaussian and we approximate Hessian by using integral image.

In the work called From Wide-baseline Point and Line Correspondences to 3D, Herbert Bay says that:

Gaussians are optimal for scale-space analysis [Koenderink 1984, Lindeberg 1990]

So explanation supposed to be found somewhere in the articles:

[Lindeberg 1990] T. Lindeberg. Scale-space for discrete signals. PAMI, 12(3):234–254, 1990. 2.3

[Koenderink 1984] J.J. Koenderink. The structure of images. Biological Cy- bernetics, 50:363 – 370, 1984. 2.3, 2.4

Also, you can implement mixed algorithm using DoG feature detector and SURF descriptors, if you will.