6
$\begingroup$

The SIFT paper and the paper of Anatomy of the SIFT Method do not mention that the input images should be preprocessed (normalized, re-scaled) before feeding images into the standard SIFT algorithm. But the following two SIFT implementations both normalized images to 0-1 beforehand.

  1. The implementation from the author of Anatomy of the SIFT Method
  2. Another implementation of C++ language

In both implementations above, it looks like they accept unsigned 8bits png images as input. And the statement of x /= 255 or x = x / 255 will normalize the pixel value to 0-1.

My questions are:

  1. Is it necessary to normalize images to 0-1 before using the standard SIFT algorithm and the standard/default parameters described in the papers?

  2. I am guessing here that if I want to use the default parameters described in the paper then I have to normalize it to 0-1. A little experiment is done here.

  • An unsigned 8-bits image is normalized to 0-1: 796 keypoints found with the standard SIFT and parameters.
  • An unsigned 8-bits image as-is: 2654 keypoints found with the standard SIFT and parameters. I can't really explain the experiment result. And the papers don't mention the prerequisite of using the standard SIFT and the default parameters.
  1. I've been using the standard SIFT and the default parameters processing unsigned 12-bits images and I found that more keypoints are found if images are not normalized to 0-1. This is actually good result but I am just afraid that I am doing things wrong and being lucky here.

  2. In SIFT, there is one step: discarding low contrasted extrema with a threshold $C_{DOG}$. It looks like because it's using the inverse of hessian matrix, the range of the original pixel values shouldn't matter here?

Thanks for reading. Any feedback is appreciated.

Update:
A python implementation doesn't rescale the input images but rescale pixel cubes until the step of localizing scale-space extrema.

Another update: Here is an intuitive explanation of why normalization is essential in computer vision, using template matching as an example. Suppose we want to match a template (a circle) with two different shapes, one is a very bright rectangle, and another one is a very dark circle. We multiply our template with two different shapes and see which produces the biggest number. Without normalization, apparently the very bright rectangle will produce higher value than the dark circle even though the form doesn't match at all. But with normalization (variance=1), we can find the right match because the bright rectangle loses its advantage on pixel intensity. It also means that the normalization shall be performed on each patch in an image if patches are fed into algorithms one after one. It cannot perform normalization on the whole image and send patches into algorithms, which will still give advantage to local bright pixels. How about normalization range? Does it matter if it's 0 to 1 or -1 to 1

$\endgroup$

1 Answer 1

2
$\begingroup$

Scaling images into the [0, 1] range makes many operations more natural when using images. It also normalizes hyper parameters such as threshold independently of the image source.

This is the reason why many image processing algorithms starts by adjusting the image into [0, 1]. It also means that Float32 or Float64 representation will be used which means the calculations will be more accurate.

In the SIFT we do many operations (Multi Scale, Angle calculation, etc...) which require the higher precision and usually using building blocks which assume the [0, 1] range.

If you converted into FP yet didn't scale the values some values will be higher than expected (Also higher for the same image if only the number of bits used for is representation is different) which might cause to some thresholds not to be optimized.

Counting the number of key points isn't the right measure here as you're after the quality. Because the issue, usually, in the context of stereo vision, is not the number of key points or even the matching but the correctness of the matching. Hence you want the good key points.

If the algorithm was programmed with the [0, 1] it is better used that way. This is an implementation dealt and not the algorithm hence it is not mentioned in the paper.

$\endgroup$
5
  • 1
    $\begingroup$ Hi Royi: I agree with you that scaling images into [0,1] range is a norm in CV world. However, I think a paper should still mention the preprocessing of an image if it provides default parameters to readers. Otherwise readers can only guess how to make it work. Yeah, in my case, more number of keypoints (kps) is found without using scaling, but also the SIFT descriptor works so well that more number of kps generated means more number of correctly matching kps pairs. In contrast, if applying scaling to images, it will generate fewer kps and results to less correctly matching kps. Thanks~ $\endgroup$
    – Lion Lai
    Nov 8, 2022 at 3:57
  • $\begingroup$ @LionLai, I agree. It might be. It might even suggest on implementation bug. But all those are implementation details. The input of data should match the assumptions of the implementation, not only the abstract algorithm. $\endgroup$
    – Royi
    Nov 8, 2022 at 6:23
  • $\begingroup$ I really appreciate your answer and explanation. But at the same time, I still feel something is missing here. I am expecting an answer that can explain things in more detail and theoretical way. Eg, why the the default threshold C_DOG only works when pixels are rescaled? or why the result of my experiment is better if without using rescale? they are quite stern questions and probably can only be answered by the author of SIFT... $\endgroup$
    – Lion Lai
    Nov 22, 2022 at 3:09
  • $\begingroup$ I think I explained that. There is no "works only rescaled". You need to think about: what are the assumption when building it. Some of the assumptions are not useful per se, they are just conventions. One of them is working on image in the [0, 1] range. One could build the whole algorithm for the range [0, 3] if they wanted to as well. I am not sure you experiment is better or worse, but what I can say is that your change made some of the parameters which are hard coded not to be suitable / optimized. $\endgroup$
    – Royi
    Nov 22, 2022 at 13:36
  • $\begingroup$ It could be that they weren't optimized before as well and better suited to your values, yet usually such coincident is very rare. $\endgroup$
    – Royi
    Nov 22, 2022 at 13:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.