15
$\begingroup$

Image registration algorithms are usually based on point features such as SIFT (Scale-Invariant Feature Transform).

I saw some references to line features, but I was wondering if it would be possible to match image segments instead of points. For example, given source and transformed image:

enter image description here

I can do edge detection, blurring and Watershed Transform on each:

enter image description here

Regrettably, the segmentation turned out to be too different on each image to match individual segments.

I saw some papers on matching shapes and shape descriptors which are invariant to affine transforms, so this area seems to be promising...

Are there any segmentation methods more robust to affine (or even projective) deformations of the image?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ My common sense tells me that smaller regions are more robust to global transformations. Thus, the segmentation should have a lot of small segments. Also, some particular shapes are invariant to some transformations, (like circles to rotations) $\endgroup$
    – Andrey Rubshtein
    Oct 2, 2012 at 0:59
  • $\begingroup$ MSER (Maximally stable extremal regions) are regions, not points. And they're invariant to affine transformation. But it's not a segmentation method, strictly speaking. $\endgroup$
    – Niki Estner
    Oct 8, 2012 at 9:43
  • $\begingroup$ @nikie If you put your comment as answer, I would accept it. I was interested in segmentation since region features contain some information about image transformation and could be possibly used for guessing the transform between images. I will certainly study the paper about MSER. $\endgroup$
    – Libor
    Oct 8, 2012 at 11:22
  • $\begingroup$ I am currently working on CBIR using Component Trees. The Component Tree representation of an image would not depend so much on the deformations (even projective) to the image, different levels would allow comparisons and operations up to a different level of detail, and should work better than current techniques on low-textured images. It's only a research topic for now, just started, but hopefully there's something in the approach, otherwise I would not be given a grant to do this. But, if somebody else did something along these lines, might be useful. $\endgroup$
    – penelope
    Oct 9, 2012 at 8:37
  • $\begingroup$ @penelope These works on CBIR may also be useful for image mosaicing (my specific interest) where we have set of images with similar features. The current popular approach is high dimensional search over point descriptors (e.g. SIFT), which can lead to false matches between images while "regions" or "components" rather than points may be able to discriminate these. Do you have any reference to papers about Component Tree representation of images? Many thanks. $\endgroup$
    – Libor
    Oct 9, 2012 at 10:50

2 Answers 2

Reset to default
4
$\begingroup$

MSER (Maximally stable extremal regions) are regions, not points. And they're invariant to affine transformation. But it's not a segmentation method, strictly speaking

Informally speaking, the idea is to find blobs at various thresholds, then select the blobs that have the least change in shape/area over a range of thresholds. These regions should be stable for a large range of grayscale and geometric transformations.

Improve this answer
$\endgroup$
4
$\begingroup$

I am currently working on CBIR using Component Trees, which should be a relatively new idea. Some expected advantages of using Component Trees to describe images would be:

  • The Component Tree representation of an image would not depend so much on the deformations (even projective) to the image
  • Examining different levels of the tree would allow comparisons and operations up to a different level of detail
  • Discrimination and description should work better than current techniques on low-textured images.

As I just started with research related to this topic, I have just a vague idea of my goals: represent image with Component Tree and then compare the said Component Trees, either directly of by finding a vectorized representation. I will probably be able to say much more in a few weeks (or months), but for now I can only offer the list of papers recommended to me as an introduction to Component Trees (I did not read them yet):

I can maybe update the answer as and if I find something relevant.

Also, if your goal is to, in a way, more accurately match image regions instead of just points, because regions might be more discriminative, there was a nice suggestion in J. Sivic and A. Zisserman: "Video Google: A Text Retrieval Approach to Object Matching in Videos".

I am referring to the section dealing with Spatial Consistency, where a group of matches between feature points is accepted only if the feature points keep a similar spatial configuration in both images. Thus, matching is not only dependent on the type of feature extracted (DoG, MSER,...) or the descriptor (SIFT), but it also looks at the wider surroundings of a feature point, making it (at least a little) region dependent.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.