The following is from Lowe 2004 paper ( http://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf ).
One obvious approach would be to sample the local image intensities around the keypoint at the appropriate scale, and to match these using a normalized correlation measure. However, simple correlation of image patches is highly sensitive to changes that cause misregistration of samples, such as afﬁne or 3D viewpoint change or non-rigid deformations. A better approach has been demonstrated by Edelman, Intrator, and Poggio (1997). Their proposed representation was based upon a model of biological vision, in particular of complex neurons in primary visual cortex. These complex neurons respond to a gradient at a particular orientation and spatial frequency, but the location of the gradient on the retina is allowed to shift over a small receptive ﬁeld rather than being precisely localized. Edelman et al. hypothesized that the function of these complex neurons was to allow for matching and recognition of 3D objects from a range of viewpoints.
I am trying to understand SIFT descriptor. I understand the previous stage (keypoint detector).
I don't know why it is implemented that way. I want to know the story behind the story.