0
$\begingroup$

I have several circle detectors. I want to evaluate their performance in finding the spherical steel markers in X-ray images. The radii of the circles are distributed in a narrow range around 8 px.

It is important that all markers are detected, it is not so important that there are no false positives.

So the first thing that comes to mind is AUC of precision-recall curves as a measure of goodness.

There is however a problem: some detectors return a probability map of some point being a center of the circle and have up to two parameters, while others operate on edge images and return circle center coordinates. Moreover one of the latter ones has many parameters to tweak. In order to have many points on the precision-recall curve for this detector I would have to sample the multidimensional parameter space.

Although technically it is not a problem I have reservations as of conceptual validity of this, since there is some a priori knowledge of reasonable values of those parameters and also they could be optimized by using a training set of images. This renders irrelevant the precision-recall curve points that were obtained with nonsense parameter, but these points nevertheless influence the AUC. Continuing this line of thought leads to a cross-validation approach, which I've never seen being used in articles on circle detection.

What is the correct way of evaluating circle detectors for a given task.

P.S. Also asked this on cross-validated stack exchange, since the question fits somewhere in between that an this community.

$\endgroup$
3
  • $\begingroup$ Welcome to DSP.SE! The *.SE network prefers that people do not cross-post questions. The idea is that if a question is a better fit for another *.SE site from the one it was posted on, then it gets migrated there. $\endgroup$
    – Peter K.
    Commented Oct 14, 2015 at 12:54
  • $\begingroup$ @PeterK. Thank you for explanation. I noticed some question authors exercising this practice and thought it was a way to go. BTW I see that there is less attention to the question in this community than in the cross-validated SE. Do you think I should remove the question here? $\endgroup$ Commented Oct 15, 2015 at 9:40
  • $\begingroup$ Feel free to delete it if the other forum is a better fit. $\endgroup$
    – Peter K.
    Commented Oct 15, 2015 at 9:45

1 Answer 1

-1
$\begingroup$

You should use segmentation score on the whole circle as it was filled. Simplest one being Intersection-Over-Union.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.