I am trying to understand how SURF extracts features using a Hessian matrix. What I am a bit confused about, is why a second derivative Gaussian filter is used, rather than a standard Gaussian filter? What effect does this have on the system?
First of all, SURF only uses it as a blob detector to find interest points. The first order derivatives only give you directional information such as edge-ness. However, the Hessian (as the matrix of second derivatives) contains local structure information such as curvature / concavity etc. For instance, its eigenvectors can be used for cornerness measures. It is just more informative. Also, its determinant is used as an automatic scale selector.
Another common alternative would be a Laplacian detector, but Hessian shows better scale selection properties.
SURF detector is kind of blob detector which find the local maxima of DoH (determinant of Hessian) over scale space to find both coordinate and the scale of feature point.
Hessian is the matrix of second order derivatives. To estimate Hessian at different scales we have to perform convolution with Gaussian to obtain image at different scales also take the derivative by finite difference approximations. Now considering convolutions and taking derivatives are linear operator we could interchange their order or combine them, so instead of performing these operations seperately we perform the combination of these two operation by applying the derivatives o Gaussian as convolution kernel.
By the way in SURF method to speed up calculations we don't use Gaussian and we approximate Hessian by using integral image.
In the work called From Wide-baseline Point and Line Correspondences to 3D, Herbert Bay says that:
Gaussians are optimal for scale-space analysis [Koenderink 1984, Lindeberg 1990]
So explanation supposed to be found somewhere in the articles:
[Lindeberg 1990] T. Lindeberg. Scale-space for discrete signals. PAMI, 12(3):234–254, 1990. 2.3
[Koenderink 1984] J.J. Koenderink. The structure of images. Biological Cy- bernetics, 50:363 – 370, 1984. 2.3, 2.4
Also, you can implement mixed algorithm using DoG feature detector and SURF descriptors, if you will.