2
$\begingroup$

I am working on an image segmentation project to identify playing cards on varying backgrounds. For my use case I care most that I accurately extract all the cards in the image; false-positives are inconvenient but not fatal. I also care that I can run the segmentation without user interaction (i.e., an algorithm like GrabCut does not work for me).

So far I have the following simple algorithm based on Canny edge detection and contour selection:

# Pre-processing: Convert frame to standard size, 1024x768
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
gray = cv2.medianBlur(gray, 9)
edges = cv2.Canny(gray, 10, 25)
_, contours, _ = cv2.findContours(edges, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
for c in contours:
    rect = cv2.minAreaRect(c)
    box = cv2.boxPoints(rect)
    area = cv2.contourArea(box)
    ratio = area / size
    if ratio < 0.015: # Any contour large enough is a candidate
        continue
    # Mark this box as possible card

For some (many) inputs it works well enough, or even very well:

good example

However for other inputs the contour detection does not successfully disambiguate cards. The contours are grouped in ways that link cards or result in undesirable bounding boxes:

bad example

For the record, here are some other things I have considered:

  • Using a bilateral filter instead of median blur. Might have improved results a bit, e.g., in the bad example above the first column third row card is now correctly segmented.
  • Using histogram equalization to enhance contrast. Generally obscured some necessary edges.
  • Attempting to automatically determine Canny thresholds for each image based on the median. Generally this approach missed many of the noisy card edges that we need to consider.

    def auto_canny(image, sigma=0.5):
        v = np.median(image)
        lower = int(max(0, (1.0 - sigma) * v))
        upper = int(min(255, (1.0 + sigma) * v))
        return cv2.Canny(image, lower, upper)

  • Attempting to use watershed segmentation. The fundamental problem is automatically determining the initial segmentation markers, which turns out to be roughly equivalent to identifying the cards in the first place. Finding "white" is particularly hard because its color varies with the ambient lighting conditions.

  • Attempting to use mean shift filtering / segmentation. I expected it might denoise the edge detection process by simplifying the image structure but it seemed to have more of a "blurring" effect, especially around reflections.

For reference (or, better yet, if you want to play along at home) I have a set of example inputs and the outputs of my current algorithm: album.

$\endgroup$
0
$\begingroup$

You might try a Hough Transform after your canny-Detection instead of contour detection. (Given that you dont have too much perspective distortion, i.e. the cards are rectangular and not trapezoidal).

Have a look at this related question and the links therein.

$\endgroup$
  • $\begingroup$ Thanks for the pointers! I've spent the last few days working through an implementation of the Hough transform for my use case. Unfortunately I've discovered that the edge detection appears to be too noisy to produce particularly good results (that, or my implementation is flawed, which is perhaps more likely). I'll keep poking at it because conceptually it seems like a powerful & relevant approach, but given my lack of success so far I'm happy to hear other options too. $\endgroup$ – errcw Feb 4 '17 at 2:49
  • $\begingroup$ @errcw there are various tested implementations for the hough-Transform for lines. Maybe, you can double-check your implementation against these. $\endgroup$ – Maximilian Matthé Feb 4 '17 at 15:28
  • $\begingroup$ I'm using OpenCV for the Hough transform so I expect that aspect is correct. What I am less certain of is how to best use the result. For my example images I find the edge detection is either very noisy and produces too many lines to be useful, or too strict and fails to identify critical card edges. Moreover, even when all the necessary edges are present, I failed to find Hough parameters that consistently identified all the card edge lines (especially in the presence of artifacts like glare obscuring edges). $\endgroup$ – errcw Feb 4 '17 at 21:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.