2
$\begingroup$

I'm working with a grey-scale data-collection of Hebraic tombstones in content-based image retrieval CBIR). For example an image like this tombstone. I'm going to reduce the content of the image to the centered tombstone. And after some other processing steps I'm trying to find a suitable representation of this image to enable CBIR. So far SIFT and SURF were good detectors but others like MSER fail to produce good results. The problem is the heterogeneity of the data-collection (6.700 picture of different tombstone with different shapes, symbols, ornaments and Hebraic letters).

Does anyone can give me a hint which pattern-recognition approach might be suitable? OCR won't do.

Thank for your help! If you need more picture, take a look around at the institue of German-Jewish history.

|improve this question
$\endgroup$
  • 1
    $\begingroup$ How are you planning do the retrieval? What would be the query? What would be the results? Are there multiple images of the same gravestone? $\endgroup$ – Dima Nov 16 '12 at 16:10
  • $\begingroup$ How are you planning do the retrieval? - Please be more specific. The query would be an image. The results should be the best k-means matched images using a descriptor. I'm using flann. There can be multiple images but not often. $\endgroup$ – Mr.Mountain Nov 16 '12 at 16:13
  • $\begingroup$ I am trying to understand what the results of a query are supposed to mean. If your query is an image of a gravestone from your data set, the result would be an image of a similar gravestone. What does it mean when two images of gravestones are similar? Are they two images of the same gravestone? Are the gravestones from the same cemetery? The same time period? What are the high-level questions that your system needs to answer? $\endgroup$ – Dima Nov 16 '12 at 16:44
  • $\begingroup$ The gravestone could be similar in many different ways and I'm trying to find a way to get a lot of interest points as I can from an image. From shape to epigraphics - everything could be important. I need a detektor getting the most stable features out of an image. If two gravestone are similar an expert-user for Jewish-history or historic-arts should be able to analyse it further. For this I will get a ground-truth soon. This work is going to be an exploration and evaluation of different approaches. $\endgroup$ – Mr.Mountain Nov 17 '12 at 11:38
2
$\begingroup$

From what you said in the comments, it seems that CBIR is not really what you need here. I am guessing that you would help the historians more by clustering the images, rather than by doing retrieval.

SIFT and SURF are a good place to start. You might also try the FREAK descriptor, which you can compute at the points detected by SIFT or SURF.

Then the important question is what you do with all those features. For a problem like this you should consider the bag-of-features approach or the pyramid match kernel.

Edit: you may be right to look for a different interest point detector. SIFT and SURF detectors give you centers of blobs (roughly circular patches of uniform intensity). Since you are looking at man-made gravestones with Hebrew text on them, corners might work better. A classic approach is the Harris-Laplacian corner detector. Note that the original Harris detector is not multi-scale. For this application you definitely need multiple scales. Other corner detectors such as FAST and AGAST have been proposed recently. I have heard good things about AGAST, but I have not tried it.

|improve this answer
$\endgroup$
  • 1
    $\begingroup$ +1 for FREAK descriptor. Very interesting, didn't know about this one. $\endgroup$ – Andrey Rubshtein Nov 18 '12 at 10:00
  • $\begingroup$ Oh - Yes. Now I understand what you've meant with "reatrieval". I want to use detectors and clustering. But this is still called cbir. The other concepts are used for this purpose. I know the SIFT/SURF and FREAK. There is much more like SUSAN, MSER, BRIEF etc. I'm searching for a lokal-feature salient-point-like approach. $\endgroup$ – Mr.Mountain Nov 18 '12 at 10:52
  • $\begingroup$ Interesting... CBIR is content-based image retrieval. I would think that clustering is distinct from CBIR, although it can be used as a building block for CBIR... Anyway, I am being picky about terminology here. :) $\endgroup$ – Dima Nov 18 '12 at 17:59
  • $\begingroup$ @Andrey, it is new. It has just been published in the last CVPR, and it is now in OpenCV. It got an award at CVPR too. $\endgroup$ – Dima Nov 18 '12 at 18:18
  • $\begingroup$ You're right. Clustering can be distinct. =) I' will try AGAST, SURF, SIFT and perhaps 2-3 other detectors (perhaps Harris-Affine/Harris-Laplace). MSER failed at a first test with sift-descriptor and FLANN. FREAK will definitely be evaluated for this CBIR-purpose. If you know any other good detector or descriptor please comment. Thank you a lot for your help. $\endgroup$ – Mr.Mountain Nov 19 '12 at 20:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.