Determinant of Hessian approximation (SURF)

Question

I have a question regarding formula in SURF article by Bay et al.

Theory

Given a point $p=(x,y)$ in an image $I$, the Hessian matrix $\mathcal{H}$ in $x$ at scale $\sigma$ is defined as follows

$$ \mathcal{H}(p, \sigma) = \begin{bmatrix}L_{xx}(p, \sigma) & L_{xy}(p, \sigma)\\L_{xy}(p, \sigma)& L_{yy}(p, \sigma)\end{bmatrix}, $$

where $L_{xx}(p,\sigma)$ is the convolution of the Gaussian second order derivative $\frac{\partial^2}{\partial x^2}g(\sigma)$ with the image $I$ in point p, and similarly for $L_{xy}(p,\sigma), L_{yy}(p,\sigma)$.

The $9\times9$ box filters in Fig. 2 are approximations of a Gaussian with $\sigma=1.2$ and represent the lowest scale (i.e. highest spatial resolution) for computing the blob response maps. We will denote them by $D_{xx}$, $D_{yy}$, and $D_{xy}$. The weights applied to the rectangular regions are kept simple for computational efficiency. This yields $$ \det(\mathcal{H}_{approx}) = D_{xx}D_{yy} - (wD_{xy})^2. $$

The relative weight $w$ of the filter responses is used to balance the expression for the Hessian’s determinant. This is needed for the energy conservation between the Gaussian kernels and the approximated Gaussian kernels, $$ w = \frac{\big\lvert L_{xy}(1.2)\big\rvert_F\big\lvert D_{yy}(9)\big\rvert_F}{\big\lvert L_{yy}(1.2)\big\rvert_F\big\lvert D_{xy}(9)\big\rvert_F} = 0.912\ldots \approx 0.9, $$

where $|X|_F$ is the Frobenius norm. Notice that for theoretical correctness, the weighting changes depending on the scale. In practice, we keep this factor constant, as this did not have a significant impact on the results in our experiments.

Questions

• Main question is how this formula for $w$ is obtained?
• And my another concern is to why $w$ is within the brackets with $D_{xy}$. If it's just a balancing coefficient wouldn't it make more sense to write like this $wD_{xy}^2$? So I guess there is some motivation behind it.

My general purpose is to generalize this method for 3D, so it would be great if anyone could share some thoughts about balancing coefficients in that case or some useful links containing relevant information.

Answer 1

1. I think $w$ is a factor that makes the ratio of $D_{xy}$ to $D_{yy}$ the same as $L_{xy}$ to $L_{yy}$. This is so the value of the determinant for the simplified kernel roughly matches that of the continuous version.

2. You could write it that way. But if you leave it as $wD_{xy}$ then if used in other expressions you don't have to muck around with adding square roots. e.g. $(wD_{xx})^3$ expressed your suggested way would be $\sqrt{w}^3D_{xx}^3$