0
$\begingroup$

I'm working on some edge detection algorithms that I want to evaluate them. I want to know what are standard ways to evaluate them that check does them work better that old ones or no.

thank you for helping

$\endgroup$

closed as too broad by Laurent Duval, MBaz, Peter K. Aug 30 '17 at 12:20

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

1
$\begingroup$

Something you could do to evaluate your solution's performance is to design a pipeline in which your home-made algorithm is applied to an image, and then you apply to its output any method aiming to detect objects or labels in the image, e.g. an active contour or a labelling. Do the same for the edge-detection filters you want to compare to in order to get qualitative results, then metrics computing such as Dice or Jaccard coefficients will give you an idea of the quantitative results of your method. In medical picture processing, metrics computing is what is used to compare an segmentation algorithm's performance to the ground truth, i.e. the segmentation that was human-performed, so I guess it is precise enough for what you want to do.

However, I would like to say that measuring performance of a picture-processing algorithm is something difficult to do and sometimes "meaningless": for instance, this discussion on ITK forum explains why there is no such thing as the "right output" of a Canny filter. It all depends on which objects you want to focus on in your image.

Regards.

$\endgroup$
1
$\begingroup$

There are many factors to evaluate. for example, you can compare a number of correct vs incorrect edge, how much your algorithm faster than other work to get same (or close) result.

If dataset available you can compare your result with other works directly without needs to run other people work but if no standard dataset you have to run other edge algorithms on the same dataset to compare it with your result.

For more information please refer to edges and arcs detection paper and if you need the code the author will send it to you.

$\endgroup$
1
$\begingroup$

I strongly agree with Ava. Without a clear formulation of your task, you cannot make a meaningful ranking of algorithms.

Think about your most important task and formulate clear goals. Here is an example of such an investigation:

"Detect only the outer edges of a shoe box in grey value images in a short amount of time".

Goal 1: Detect only the outer edges --> measure number of connected components, if 1, check if it is the contour of the parcel, if !=1, vary parameters and try again.

Goal 2: Be fast --> If goal 1 is reached, measure time needed to compute these edges.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.