As far as I understand what you call defects are detcted by changes in color of the fruit.
I would proceed as follow:
- edge detection algorithm, (as you did), to detect the fruit region in the image.
- In this region, compute the median color, that is the color that matches the most pixels (for example, it would be some kind of yellow for a banana)
- For each pixel in the fruit region, check the distant between the median computed at step 2 and the color of the pixel. If the distance is high, mark the pixel as defect
- At the end, apply some morphological operator to remove outliers
Hope this helps.
