As far as I understand what you call defects are detcted by changes in color of the fruit.

I would proceed as follow:

1. edge detection algorithm, (as you did), to detect the fruit region in the image.
2. In this region, compute the median color, that is the color that matches the most pixels (for example, it would be some kind of yellow for a banana)
3. For each pixel in the fruit region, check the distant between the median computed at step 2 and the color of the pixel. If the distance is high, mark the pixel as defect
4. At the end, apply some morphological operator to remove outliers

Hope this helps.

As far as I understand what you call defects are detcted by changes in color of the fruit.

I would proceed as follow:

1. edge detection algorithm, (as you did), to detect the fruit region in the image.
2. In this region, compute the median color, that is the color that matches the most pixels (for example, it would be some kind of yellow for a banana)
3. For each pixel in the fruit region, check the distant between the median computed at step 2 and the color of the pixel. If the distance is high, mark the pixel as defect
4. At the end, apply some morphological operator to remove outliers

Hope this helps.