Gaussian pyramid: why needs the image to be downsampled

Question

In the context of a gaussian pyramid, why is the image downsampled separately although the numbers of pixels are decreased through smoothing already? In my understanding, the number of pixels will decrease due to the smoothing process already, if a gaussian filter is used within the image boundaries. An image with 8 x 5 pixels smoothed with a gaussian filter of 3 x 2 pixels will create a 6 x 4 pixel image if the filter can only be slided within the boundaries of the original image without crossing the edges.

So if the constraint of not crossing the edges of the image is applied, why would one use additional downsampling?

Answer 1

In the context of a Gaussian pyramid, why is the image down-sampled separately although the numbers of pixels are decreased through smoothing already?

After filtering the image the number of pixels are not reduced. The result would have (almost) the same size of original image.

In pyramids, after low pass filtering, the neighboring pixels all would have almost the same value and so in terms of image features, they bring no new information. On the other hand through down sampling, the computation is significantly accelerated. For example, I have tried SIFT algorithm without down sampling the DOGs, the same result were achieved but my implementation was way more slower.

Answer 2

In the context of a gaussian pyramid, why is the image downsampled separately although the numbers of pixels are decreased through smoothing already?

The Gaussian filter adjusts the bandwidth of the content of the image. The downsampling adjusts the spatial resolution of the image.

The Gaussian filter is a low pass filter. After its application to an image, only the frequencies within its pass band will be present in the resulting image.

An image of dimensions $M \times N$ pixels can depict $\frac{M}{2} \times \frac{N}{2}$ lines per pixel of spatial frequencies.

If you apply a low pass filter at $\frac{M}{4}, \frac{N}{4}$, you have reduced the spatial frequency requirements for that signal by half. Where before you needed $\frac{M}{2} \times \frac{N}{2}$ lines per pixel, now you know that spatial frequencies in your image do not exceed $\frac{M}{4}, \frac{N}{4}$ lines per pixel. The rest of the bandwidth that an image of that resolution can support goes unused.

At that point, you can reduce the image resolution to accommodate the "new" limits of the bandwidth without any loss.

From another point of view, you use the low pass filter to ensure that the resulting image will not contain any spatial frequency that could be aliased if it was to be plainly subsampled (because ultimately, resizing an image means that it will have to be resampled and "fit" into a smaller space. This smaller space means less bandwidth). Once you ensure that no frequency will be aliased, then it is safe to resize.

An image with 8 x 5 pixels smoothed with a gaussian filter of 3 x 2 pixels will create a 6 x 4 pixel image if the filter can only be slided within the boundaries of the original image without crossing the edges.

This is to do with choosing only the "valid" values in the output of the filter. Values for which all the weights of the two dimensional impulse response are applied to image pixels.

Hope this helps.