9
$\begingroup$

I have implemented a feature detector based on Harris corners. It works fine most times, but there are cases where it performs poorly. I need to make it work on many different images without configuring it individually.

The problem is with the detector threshold value. If set too low, the detector fires too many times resulting in huge number of features. If set too high, there are too less features.

I have partially resolved this by ANMS (Adaptive Non-Maximal Suppression) to reduce number of features before assigning a descriptor vectors to them.

However, images like this one are the problem:

enter image description here

They have low contrast and I cannot "afford" setting threshold too low for all images. It would cause detector to work on these images, but other images would contain hundreths of thousands of features, which will be slow to filter with ANMS and this will harm overall performance.

I was thinking of adjusting the image before feature detection. Maybe histogram equalization would do the job. This may be a valid operation since global change of contrast have no effect on feature descriptors (they are invariant to change in brightness and contrast).

Maybe working with adaptive threshold or some heuristic would work better.

Any other suggestions?

Improve this question
$\endgroup$
6
  • $\begingroup$ @Seyhmus Güngören: Oh. I have accepted the answers I got so far since I found satisfactory enough solutions, but still waiting for better or more elaborate answers. $\endgroup$
    – Libor
    Commented Aug 23, 2012 at 18:11
  • $\begingroup$ @Libor It is good for your new questions because it might be more appealing to solve your questions in the current case. $\endgroup$
    – Seyhmus Güngören
    Commented Aug 23, 2012 at 18:18
  • $\begingroup$ @Libor did you consider histogram matching choosing a gaussian like histogram which has a mean around 128? $\endgroup$
    – Seyhmus Güngören
    Commented Aug 26, 2012 at 17:07
  • $\begingroup$ @SeyhmusGüngören Yes I was thinking about this. I will probably try that with few other ideas. $\endgroup$
    – Libor
    Commented Aug 26, 2012 at 17:34
  • $\begingroup$ @Libro Can you please say how you plan to use the detected features? My only ideas are content based image retrieval or matching (e.g. for homography estimation), but since that's the only thing I've ever done with them I might be wrong :) If, on the other hand, that is what you plan to use them for, I might be able to contribute. $\endgroup$
    – penelope
    Commented Sep 22, 2012 at 16:23

2 Answers 2

Reset to default
1
$\begingroup$

A possibility would be to do a simple edge detection (such as Laplace), and use the mean intensity of the result as a basis for the threshold for the Harris corners. When you have low contrast, you will get less edges and with lower intensity, with high contrast you get more edges and with higher intensities.

You are not the only one struggling with this problem. If you have access to paper databases, this might be interesting:

It might be worth to search for (auto) adaptive harris corner detection more.

Improve this answer
$\endgroup$
4
  • $\begingroup$ That's not a contradiction. The detector have two stages: 1) detect features, 2) describe features. The histogram equalization should have effect on stage 1 (more features detected), not on stage 2. I need a moderate amount of features to be detected, so they need not to be filtered much. $\endgroup$
    – Libor
    Commented Aug 23, 2012 at 11:14
  • $\begingroup$ Ok, I thought of this, but it wasn't completely clear from your question. I do not mean to use the edge image as input for the feature detector, but as your own measure of what the value for the threshold should be. $\endgroup$
    – Geerten
    Commented Aug 23, 2012 at 11:21
  • $\begingroup$ Thanks, that's an interesting thought. The Harris corner detector uses derivative images (dx, dy, dxy) to construct corner measure at each point. Since this is already based on edge measures, I am now thinking about taking histogram of the corner response function and computing threshold of that histogram. You inspired me, thanks :) $\endgroup$
    – Libor
    Commented Aug 23, 2012 at 14:04
  • $\begingroup$ I'm glad I could inspire you ;) Added a paper that might be helpful. $\endgroup$
    – Geerten
    Commented Aug 23, 2012 at 14:21
2
$\begingroup$

Do you really have to use Harris corners? There is many features developed after Harris corners, with better properties. A good overview can be found in this article:

Based on that article as well as my personal experience, I would suggest switching either to MSER (Maximally Stable Extermal Regions), or even combine them with DoG (Difference of Gaussians) -- the features first presented as a part of the SIFT pipeline.

If the problem really is in low contrast, then MSER features should really make you happy: they are (fairly) invariant to changes in lighting. In short, they are connected regions of the image stable through a series of different threshold binarizations.

The feature extraction process is independent from calculating the descriptors, so it shouldn't be too hard to integrate new ways of feature extraction in to your process.

Also, I've heard of (but never actually worked with) Multiscale Harris corners as an extension to Harris corners. I don't know much about them and personally can not recommend any reading materials on this topic, so I leave article search and picking the most interesting materials to you.

Furthermore, might I suggest that the image you posted might have other problems than low contrast. In my personal experience, vegetation like bushes or possibly the field you have, as well as the lovely bubbly clouds tend to produce "generic features" -- features which tend to have equally similar (or dissimilar) descriptors as a lot of other features.

Practically, this means that when doing feature matching on two images from a different perspective, features extracted from these kinds of surfaces tended to be falsely matched. I have done a Master thesis that in a large part deals with feature extraction to be used in feature matching further used to calculate a homography transformation between two images when I came across this problem. I didn't find any other articles describing this problem at the time, but my thesis might be helpful for your overall approach.

Lastly, as you have set, thresholds and techniques that work just fine on most images extract to little features in this kind of images, because of its mostly homogeneous areas. This kind of images present problems in feature matching (which can be extended to image stitching), content based image retrieval, and I would presume tracking as well as similar applications. No method currently works quite well on them.

Methods that work good on this kind of images as well as the typical cases are being explored and researched currently, such as an approach I started working on briefly described in this answer.

Improve this answer
$\endgroup$
8
  • $\begingroup$ Thanks for the detailed answer, I will go through the papers when having some spare time. I was implementing a feature detector facing two problems: complexity of implementation and patent issues. My application is a commercial image alignment and stitching library and so I have limited resources and time for implementation and cannot afford paying for SIFT or SURF. I will probably switch to either MSER or other advanced detector/descriptor, but so far Harris corners work well with the exception of images with bad lighting. $\endgroup$
    – Libor
    Commented Oct 14, 2012 at 20:50
  • $\begingroup$ @Libor That's the beauty of it: You don't have to switch. You can just add the new features in to your existing detection->description pipeline. No matter how the features are extracted, you can always calculate their descriptors with a same tactic. Of all the stuff I've written, maybe the first article mentioned comparing various options for detection/description might prove the most useful. $\endgroup$
    – penelope
    Commented Oct 14, 2012 at 21:01
  • $\begingroup$ I used gather large descriptors and then used PCA to improve speed and discriminative power of the descriptors. The PCA, however, is quite costly for large datasets. This work attracted me because of generic enhancement of the descriptors. So far all I use is the "Feature Space Outlier Rejection", which is simply thresholding on feature matches based on 1-nn/2-nn distance. This is described by D. Lowe in his papers and have a very good discriminative power as it exploits the shell property of distances in high-dim spaces. $\endgroup$
    – Libor
    Commented Oct 14, 2012 at 22:11
  • $\begingroup$ As for the detectors, large viewpoint changes and scale invariance is not a problem, as with image mosaicing (panoramas, microscopes) the zoom is usually kept unchanging and affine or projective deformations are quite small between matching images. The main problem is really too less or too many features detected and poor descriptors. $\endgroup$
    – Libor
    Commented Oct 14, 2012 at 22:14
  • $\begingroup$ I personally don't know much about descriptor choice, I've worked only with SIFT. But the link you provided mentioned that they are similar to DAISY descriptors, which I also remember being evaluated as very good. Combining more feature extractors should hopefully provide you with more features, and scale invariance can only be a plus, even if you don't need it. I read some works mentioning that working with multiple feature extractors increases discriminative powers (I can look up the links if you want). $\endgroup$
    – penelope
    Commented Oct 15, 2012 at 18:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.