I'm working on some edge detection algorithms that I want to evaluate them. I want to know what are standard ways to evaluate them that check does them work better that old ones or no.
thank you for helping
Signal Processing Stack Exchange is a question and answer site for practitioners of the art and science of signal, image and video processing. It only takes a minute to sign up.Sign up to join this community
Something you could do to evaluate your solution's performance is to design a pipeline in which your home-made algorithm is applied to an image, and then you apply to its output any method aiming to detect objects or labels in the image, e.g. an active contour or a labelling. Do the same for the edge-detection filters you want to compare to in order to get qualitative results, then metrics computing such as Dice or Jaccard coefficients will give you an idea of the quantitative results of your method. In medical picture processing, metrics computing is what is used to compare an segmentation algorithm's performance to the ground truth, i.e. the segmentation that was human-performed, so I guess it is precise enough for what you want to do.
However, I would like to say that measuring performance of a picture-processing algorithm is something difficult to do and sometimes "meaningless": for instance, this discussion on ITK forum explains why there is no such thing as the "right output" of a Canny filter. It all depends on which objects you want to focus on in your image.
I strongly agree with Ava. Without a clear formulation of your task, you cannot make a meaningful ranking of algorithms.
Think about your most important task and formulate clear goals. Here is an example of such an investigation:
"Detect only the outer edges of a shoe box in grey value images in a short amount of time".
Goal 1: Detect only the outer edges --> measure number of connected components, if 1, check if it is the contour of the parcel, if !=1, vary parameters and try again.
Goal 2: Be fast --> If goal 1 is reached, measure time needed to compute these edges.
There are many factors to evaluate. for example, you can compare a number of correct vs incorrect edge, how much your algorithm faster than other work to get same (or close) result.
If dataset available you can compare your result with other works directly without needs to run other people work but if no standard dataset you have to run other edge algorithms on the same dataset to compare it with your result.
For more information please refer to edges and arcs detection paper and if you need the code the author will send it to you.