Function for response over image scales

Question

I want to write code in matlab to find a characteristic scale of an image patch. I have an image patch X, I need to maximize function F(X,sigma) so that sigma is the characteristic sigma.

For my experiments, F can be the laplacian of the image, sigma is the "power" of a gaussian filter on the image. F(x,sigma) should return for each x (image patch) its response over scales of increasing sigmas (i.e. a discrete value for each X and sigma). My questions are:

1. The laplacian is obtained by convolving laplacian kernel with the image, where does the sigma takes place ? (is it by smoothing the image before (the laplace) with G(sigma)?)
2. How can I define F(X,sigma) to return discrete value (X is nXm), ? i.e. what does this function do exactly.

---

Dima and Jean, Thank you for answers, the reason I'm asking this question is because I need to do something similar on 3D data (probably/maybe with some other normalized derivatives), and I first want to see how it works with 2D data (i.e. image). I understand now what I need to do for constructing the scale space, there is still one thing I'm not sure of and it is finding the local maxima over F. For each image (x,y) and σ - F is actually the derivative of the image. What does is mean local maxima of F over x,y and σ ? look for local maxima(s) at each scale ? there are several local maxima at each scale, are these local maxima correspond (in position) over all scales? if so, I understand that for each such position I need to find the σ that gives best maxima over all scales.. Is this is what you meant guys ? I'm sorry if I'm going to much into details, I just want to make sure I completely understand, THANK YOU !

Answer 1

You probably want to check out the notion of scale-space and the links branching from this: http://en.wikipedia.org/wiki/Scale_space

Assuming $X$ (sorry the way the question is asked is a bit unclear to me) is the image you want to find the scale: What you probably want to start with for the $F$ function is the Laplacian of Gaussian filter. You can show that it is the optimal detector for gaussian blobs even in the presence of white noise (Daniel Sage paper)

The scale then appear with the $\sigma$ of the LoG filter. This filter will have its best response when the size of the blob you apply it on matches the filter's $\sigma$.

However, you can see with this approach that it is not enough to have a maximum of $F$ at the target scale when $\sigma$ span all possible values. This is due to the fact that when you filter, the integral of the filter comes into play and scales the response. So you want to normalize you LoG filter. You can show that the correct expression for a single blob scale detector is

$$ L(\sigma) = \sigma^2 \left( \frac{\partial^2 G}{\partial x^2} + \frac{\partial^2 G}{\partial y^2} \right) $$

where $G$ is the gaussian filter of scale $\sigma$.

---

Ok I just saw your edit of the question. I took a slightly different approach than Dima, but in the end, hope fully the results are going to be the same. I need to start over with the notations.

Let's say that $I(x,y)$ is your image, that $L(\sigma)$ is the laplacian of gaussian filter of scale $\sigma$.

If you apply $L(\sigma)$ on $I(x,y)$, you will get a new image F so that: $$ F(x,y,\sigma) = L(\sigma) \star I(x,y) $$

What is cool about $F(x,y,\sigma)$? Not much if you are interested in girls, but if you are interested in finding blobs in your image $I$, there is plenty to say. If you look at the local maxima in $(x,y)$ for a given $\sigma$, you will see that they point to the center of blobs.

Even better: let's say now that you sit on one of these local maxima on $(x,y)$ and that you let $\sigma$ vary. Well, if you used the filter $L$ with the normalization factor, you will see that the absolute value of $F(\sigma)$ actually peaks for a $\sigma$ that matches the size of your blob.

So this is how you can get the location the blobs in your image (from the $(x,y)$ local maxima) and the size of these blobs (from the $\sigma$ value at which the value of these maxima peaks).

There is still a bit to do: the optimal $\sigma$ for a blob of diameter $d$ actually depends in the dimensionality of the problem (2D vs 3D), you can retrieve the relation analytically or experimentally.

Also if you want to go for real 3D images, in most cases (at least mine...) the pixel size in X & Y is not the same in Z. It makes your round blob look like rugby ballons seen from the side. You want to deal with this by using a different sigma in the laplacian filter, accounting from this anisotropic calibration.

---

For an efficient implementation, that takes advantage of having to compute over multiple $\sigma$s, you want to check DoG filters: http://en.wikipedia.org/wiki/Difference_of_Gaussians

_{(also, it's Jean-Yves (not Jean (which is a girl name (not that I have a problem with that (I love LISP)))))}

Answer 2

Conceptually this works as follows: first you construct a Gaussian scale space by convolving the given image with the Gaussian filters of increasing $\sigma$. This is where the sigma comes from. Thus you have a 3D scale volume, where each plane is an increasingly blurred version of the original image. Then you evaluate your function $F$, which in your case would be a Laplacian, over each scale plane, i. e. at each $\sigma$. Then you find local maxima of $F$ over $x, y$, and $\sigma$.

Note that in practice you can use the fact that convolution is a linear operation in various ways. For example, you can generate each scale plane from the previous plane, rather than from the original image. This allows you to use the same filter, rather using several filters with increasing $\sigma$. Or you can generate $F(\sigma)$ directly from the original image by using the Laplacian-of-Gaussian filter of the corresponding $\sigma$