3
$\begingroup$

I am learning about SIFT detection and descriptor. I am slightly unsure about why a Gaussian pyramid is built for the image.

enter image description here

I do understand that within each octave, we are applying the Difference of Gaussian filter at different scales to the image and finding for each pixel location, whether it is a local maxima. We do this also for the image at a smaller scale. So now we have marked out the pixels in two different scales of image that have local extrema values.

What do we do with the local extrema values in these 2 different scales ? Do we compute SIFT descriptors at the 2 different sizes of image ?

Improve this question
$\endgroup$
2
  • $\begingroup$ Can you find enough details in this answer and the reference paper mentioned [Why do we need multiple layers in each octave and multiple octaves in SIFT? ](dsp.stackexchange.com/a/68584/15892) $\endgroup$
    – Laurent Duval
    Oct 28, 2020 at 8:16
  • $\begingroup$ I tried to read the reference but got lost pretty quickly.. I have been mainly watching the computer vision lecture series by UCF to understand SIFT. youtube.com/… $\endgroup$
    – calveeen
    Oct 28, 2020 at 8:34

1 Answer 1

Reset to default
3
$\begingroup$

Actually, the purpose of all this is to approximate a Laplacian of Gaussian!

This computation is part of the corner detection of SIFT. You can find corners by examining extrema of the Laplacian of Gaussians (2nd order derivative). You use Gaussians for denoising, and a Laplacian to find inflection points. However, it is classical to not deal directly with computing this second order derivative, but to approximate it instead by difference of Gaussians for simplicity. This is sometimnes referred to as Gaussian blob detection.

You can find some illustration of this process here.

With this approach your difference of Gaussians (DoG) will have a fixed (small) size in pixels, so you're bounding the computational cost. But you still want to detect blobs of different sizes (because objects have varyng size,s, and the camera can be closer or further away from the scene), and you achieve this by "zooming out the image" with the various scales of the pyramid and keeping the DoG size fixed, instead of keeping the original image size and changing the size of your blob detector.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Hi thank you for your reply. I understand this point. What I don’t understand is the reason for building a Gaussian pyramid and computing local extrema points for images at smaller and smaller scales $\endgroup$
    – calveeen
    Oct 29, 2020 at 0:29
  • $\begingroup$ (Edited the answer) $\endgroup$
    – sansuiso
    Oct 30, 2020 at 10:42
  • $\begingroup$ the DOG's are computed at different scales within an octave. doesn't this achieve a different sized blob detector ? $\endgroup$
    – calveeen
    Oct 30, 2020 at 11:17
  • $\begingroup$ In passing: hellooooo $\endgroup$
    – Laurent Duval
    Oct 30, 2020 at 11:35
  • $\begingroup$ (Hello Laurent!) Differences between octaves account for a blob size 2 x smaller along each axis (so the next blob is 4 times smaller in area). Inside a given octave you have more subtle and gradual size changes, so you can detect more sizes. Also, the intra-octave scales are useful for (1) some anti-aliasing when downsampling and (2) the blob scale refinement step. During this refinement step the exact scale where the laplacian becomes zero will be estimated based on the two scales where the Laplacian sign flips. More scales means more accuracy here. $\endgroup$
    – sansuiso
    Oct 30, 2020 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.