I have several circle detectors. I want to evaluate their performance in finding the spherical steel markers in X-ray images. The radii of the circles are distributed in a narrow range around 8 px.

It is important that all markers are detected, it is not so important that there are no false positives.

So the first thing that comes to mind is AUC of precision-recall curves as a measure of goodness.

There is however a problem: some detectors return a probability map of some point being a center of the circle and have up to two parameters, while others operate on edge images and return circle center coordinates. Moreover one of the latter ones has many parameters to tweak. In order to have many points on the precision-recall curve for this detector I would have to sample the multidimensional parameter space.

Although technically it is not a problem I have reservations as of conceptual validity of this, since there is some a priori knowledge of reasonable values of those parameters and also they could be optimized by using a training set of images. This renders irrelevant the precision-recall curve points that were obtained with nonsense parameter, but these points nevertheless influence the AUC. Continuing this line of thought leads to a cross-validation approach, which I've never seen being used in articles on circle detection.

What is the correct way of evaluating circle detectors for a given task.

P.S. Also asked this on cross-validated stack exchange, since the question fits somewhere in between that an this community.

  • $\begingroup$ Welcome to DSP.SE! The *.SE network prefers that people do not cross-post questions. The idea is that if a question is a better fit for another *.SE site from the one it was posted on, then it gets migrated there. $\endgroup$ – Peter K. Oct 14 '15 at 12:54
  • $\begingroup$ @PeterK. Thank you for explanation. I noticed some question authors exercising this practice and thought it was a way to go. BTW I see that there is less attention to the question in this community than in the cross-validated SE. Do you think I should remove the question here? $\endgroup$ – Hennadii Madan Oct 15 '15 at 9:40
  • $\begingroup$ Feel free to delete it if the other forum is a better fit. $\endgroup$ – Peter K. Oct 15 '15 at 9:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.