9
$\begingroup$

I have a question concerning object recognition, especially recognizing car-models! I am at the beginning of a work about identifying the same car-model in different images. At the moment I think one of the best algorithm for 3D object recognition is SIFT but after playing around a bit with a demo implementation I have the strange feeling this algorithm has some problems with shiny metal objects like cars, especially if they have different colors.

Does anyone know some work on this area in general some suitable algorithm for the task of finding the same car-model in different images?

Thanks in advance for your help!

|improve this question
$\endgroup$
  • 2
    $\begingroup$ Can you post some example images? $\endgroup$ – endolith Mar 20 '12 at 21:16
  • $\begingroup$ Sure. Images for creating a model of car-models ;-) could be like: s5 coupe training 1 or like s5 coupe training 2 but also 'normal' pictures. Query images could be like s5 coupe query 1 hope that helps! $\endgroup$ – jstr Mar 20 '12 at 21:39
  • $\begingroup$ What alternative Feature-Detectors like SIFT, GLOH or SURF are there to identify suitable key-points on cars? $\endgroup$ – jstr Mar 27 '12 at 7:10
  • $\begingroup$ @jstr if you ended up implementing the scheme described below, how well did it work? $\endgroup$ – solvingPuzzles Feb 22 '13 at 7:31
7
$\begingroup$

I would have a look at the so called "bag of words" or "visual words" approach. It is increasingly used for image categorization and identification. This algorithm usually starts by detecting robust points, such as SIFT points, in an image. The region around these found points (the 128 bit SIFT descriptor in your case) is used.

In the most simple form, one can collect all data from all descriptors from all images and cluster them, for example using k-means. Every original image then has descriptors that contribute to a number of clusters. The centroids of these clusters, i.e. the visual words, can be used as a new descriptor for the image. Basically you hope that the clusters an image its descriptors contribute to, is indicative of the image category.

Again, in the most simple case, you have a list of clusters, and per image, you count which of these clusters contained descriptors from that image and how many. This is similar to the Term Frequency/ Inverse Document Frequency (TD/IFD) method used in text retrieval. See this quick and dirty Matlab script.

This approach is actively researched and there are many much more advanced algorithms around.

The VLfeat website contains a nice more advanced demo of this approach, classifying the caltech 101 dataset. Also noteworthy, are results and software from Caltech itself.

|improve this answer
$\endgroup$
  • $\begingroup$ Hey Maurits, thank you for your answer. I'll think about that! But one question. If I have the 'visual words' how do I measure the distance between them? I think I would use the SIFT descriptors is that correct? - Lowe has one paper in which he describes a method for recognizing 3D objects by building up models of SIFT descriptors. Does anyone know some good other papers on this topic (3D object recognition with other features)? $\endgroup$ – jstr Mar 22 '12 at 8:06
  • $\begingroup$ In this case, just the euclidean distance, as you are clustering integer vectors. I don't think you have to measure the distance between cluster centroids per se, but rather, when presented with a query image (and thus query descriptors) you measure to which centroids these descriptors are the most closest. $\endgroup$ – Maurits Mar 22 '12 at 9:16
  • $\begingroup$ Ok using a distance measure is clear ;-) but on which data? On the SIFT Descriptors per visual word? $\endgroup$ – jstr Mar 22 '12 at 9:39
  • $\begingroup$ Three times in fact, as a metric for the initial clustering, to ascertain to which centroid/visualword a query descriptor is the most close, and then finally, to compare the query td/idf vector against the ones in the database. $\endgroup$ – Maurits Mar 22 '12 at 18:56
  • $\begingroup$ Ok I got that ;-) but on which data does the distance measure work? On the SIFT descriptors? $\endgroup$ – jstr Mar 23 '12 at 11:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.