I would have a look at the so called "bag of words" or "visual words" approach. It is increasingly used for image categorization and identification. This algorithm usually starts by detecting robust points, such as SIFT points, in an image. The region around these found points (the 128 bit SIFT descriptor in your case) is used.
In the most simple form, one can collect all data from all descriptors from all images and cluster them, for example using k-means. Every original image then has descriptors that contribute to a number of clusters. The centroids of these clusters, i.e. the visual words, can be used as a new descriptor for the image. Basically you hope that the clusters an image its descriptors contribute to, is indicative of the image category.
Again, in the most simple case, you have a list of clusters, and per image, you count which of these clusters contained descriptors from that image and how many. This is similar to the Term Frequency/ Inverse Document Frequency (TD/IFD) method used in text retrieval. See this quick and dirty Matlab script.
This approach is actively researched and there are many much more advanced algorithms around.
The VLfeat website contains a nice more advanced demo of this approach, classifying the caltech 101 dataset. Also noteworthy, are results and software from Caltech itself.