what exactly does scale mean in scale-space theory?

In image processing, scale-space is a technique to represent an image at different scales. But this really confuses me, since I always think that the word scale just means the size of an object, so the scale-space representation of an image shall have different sizes of the same image.

However, it turns out that the scale-space representation is composed of Gaussian-convolved images with different $\sigma$, the image size doesn't change at all, why? And further, how $\sigma$ in the Gaussian function determines the scale of the convolved image ?

• You might want to dig into descriptions of the image pyramid first to ground yourself first: en.wikipedia.org/wiki/Pyramid_%28image_processing%29 Sep 17 '13 at 0:01
• @Rethunk, do you mean that those Gaussian smoothed images are then downsampled into smaller size images? So these downsampled images of different sizes are different scales? Sep 17 '13 at 0:29
• That's a way to look at it, but I fear I'm leaving out some details. I would recommend digging into some texts and academic papers. For many purposes it's sufficient to work with image pyramids, but when you consider true scale space theory there's more involved. I don't recall, but the following paper may be one of the first I found a while back: citeseerx.ist.psu.edu/viewdoc/… Oct 28 '13 at 4:34
• Also, just try this and see what you think: start with an image of dimension X by Y. Convolve it with a Gaussian kernel--a 3x3 will do. The down sample the resultant image by taking every other pixel to generate an X/2 by Y/2 image. Compare that to a reduced resolution inage where you simply take every 2nd pixel without the intermediate step of the convolution with the Gaussian kernel. Oct 28 '13 at 4:38

There's an image processing algorithm called Retinex that uses scales to perform local contrast enhancement. Here's an OK presentation on the Multi-Scale Retinex algorithm for more detail.

The image size doesn't change because you are performing convolution of a Guassian kernel with the image. In practice this is generally accomplished in the frequency domain and the image is padded to avoid the effects of circular convolution. This is standard stuff in image processing where you convolve a 'kernel' against an image. It's a just averaging surround pixels with a 2-D Gaussian function instead of something like a block of ones.

I've seen the choice of sigma be a somewhat adhoc "This looks good to me" selection process. There are ranges published in a few papers that seem to work well for most images. Essentially small values preserve the detail in the image, medium values provide a mix of general detail and global energy, and large values provide more global representations of the image energy. I think something like small is 1-25, medium 25-75, and large is +75. When you convolve and sum all of the scales, with a bit more light math, you get a local contrast enhanced image. The results are pretty good. You can try it out on some images to see the result of each to get a better feel for it.

• Convolving an image with scale $\sigma$ will wash off details which are smaller than $\sigma$, right? But why not downsample the image? Sep 17 '13 at 9:05
• @loganecolss Downsampling the image is actually smoothing followed by sampling, so it doesn't matter if the image is smoothed and sampled or downsampled using responses of some smoothing filter. The 1983 paper "The Laplacian Pyramid as a Compact Image Code" discusses this in detail. Sep 17 '13 at 13:15
• @Libor, so you mean the smoothed image already has scale $\sigma$, no matter it's downsampled or not? Sep 17 '13 at 13:26
• @loganecolss I think so but I am not expert on the topic. The Burt & Adelson define REDUCE and EXPAND operations in their paper - they actually subsample and then upsample the image, then compute difference to obtain quasi band-pass image. The subsampled image contains all the information as the expanded version so from this point of view, they are equivalent (having same information content). Sep 18 '13 at 13:44
• @loganecolss Of course, the subsampling operation is accompanied with smoothing to avoid aliasing effects. Sep 18 '13 at 13:45

The basic idea in scale space is to parametrize the image value/function space with a single parameter so that one can localize or select interesting variations of the this parameter. The various images produced for different values of the parameter can represent different things, in image analysis they are simplifications of an input image. One can perform smoothing, bandpass filtering and various kinds of frequency based operations using linear scale spaces - this has been well studied. For a more general view on scale see.

In case of Gaussians, morpholigcal operators like erosions/dilations one produces simpler functions based on the concept of causality - we dont introduce new extrema. This is mathematically fixed by the semi-group structure. The space or family of functions produced is the scale space. One should note here that the scale parameter could actually be changed with a parameter for a dynamical system or even a parameter for diffusion schemes. Its the axioms that make this definition specific.

For the Gaussian we have two parameters the kernel size and standard deviation of the gaussian. Now the scale intuition is more or less right - we search different scales (simplifications) to find an object, like in SIFT and other algorithms. Thus the scale is representing the structure of the image, while the object (like in detection algorithms) is something being search for in this space.