Understanding Integration Scale in Multi-Scale Harris Corner Detector

Question

I would like to improve a simple Harris corner detector by adding more scales.

I don't need scale-invariant interest points, just to detect more points by processing more scales.

The Wikipedia says there are two sigmas involved: a derivative scale $\sigma_{D}$ and an integration scale $\sigma_{I}$.

So after computing derivative images on a given level, I smooth them with Gaussian filter of size $\sigma_{D}$, then for each location $(x,y)$ in the image I have the matrix:

$H(x,y)=\begin{pmatrix}I_{x}^{2} & I_{x}I_{y} \\ I_{x}I_{y} & I_{y}^{2} \end{pmatrix}$

where $I_{x}$ and $I_{y}$ are the smoothed images (with $\sigma_{D})$ on the respective location.

How do I apply the integration scale $\sigma_{I}$ now?

Do I have to store separate images $I_{x}^{2}$, $I_{y}^{2}$ and $I_{x}I_{y}$ and smooth these with $\sigma_{I}$ before computing the matrix and Harris corner response from it?

Answer 1

I think the first smoothing, by $\sigma_D$, is only done to get more stable derivatives whereas in the second step the convolution by a Gaussian with $\sigma_I$ is done to establish the 'scale-space' in which the operator is applied.

Ignoring $\sigma_D$, this looks like this in Matlab:

dx = [-1 0 1; -1 0 1; -1 0 1];  % Simple mask for derivative 
Ix = conv2(im, dx, 'same');     % Convolve against image
Iy = conv2(im, dx', 'same');    % Again for y-direction with transposed mask

% Gaussian filter, as defined by Kovesi
% www.csse.uwa.edu.au/~pk/Research/MatlabFns/index.html
sigma = 1;
g = fspecial('gaussian', max(1, fix(6*sigma)), sigma);  

% Determine smoothed squared image derivatives via convolution
Ix2 = conv2(Ix.^2, g, 'same'); 
Iy2 = conv2(Iy.^2, g, 'same');
Ixy = conv2(Ix.*Iy, g, 'same');

% Metric to define the corner score
cim = (Ix2.*Iy2 - Ixy.^2) - k*(Ix2 + Iy2).^2;    % Original measure
cim = (Ix2.*Iy2 - Ixy.^2)./(Ix2 + Iy2 + eps);    % Kovesi measure