OpenCV/C++ connect nearby contours based on distance between them

Question

I have to connect nearby contours in an image based on the distance between them which specifies if the contours are to be connected.

Now there's a question already on the same problem here https://stackoverflow.com/questions/8973017/opencv-c-obj-c-connect-nearby-contours but here he merges all contours into a single one. This I don't want. I don't think that there is some function in opencv for this but you can suggest an algorithm for that. My application goes like this:

I'm detecting hands, so I used a skin detection algorithm to determine them but since my skin ain't white and maybe because of lightening conditions sometime the contour breaks of at elbow. So i want the nearby contours to be connected but not all of them (because both both my hands will be there in contours.) (By hands I mean from shoulder to palm.)

Moreover I think that by using some edge detection I will get my hands boundaries and detect if some of this patch inside this boundary is detected as skin then whole region within this boundary will be detected as skin but i'm not sure how to do this part.

Any Help will be appreciated. Thanks in advance

Sample image:

In this Image I wanna connect points (8 connectivity) which are less than say 40 pixels of distance so that I'll get my left hand as a single contour

My aim is to only get hand's contour (I don't care about any other region)

Answer 1

If you are not worried about the speed or exact contour of hand, below is a simple solution.

The method is like this : You take each contour and find distance to other contours. If distance is less than 50, they are nearby and you put them together. If not, they are put as different.

So checking distance to each contour is a time consuming process. Takes a few seconds. So no way you can do it real time.

Also, to join contours, I put them in a single set and drew a convex hull for that set. So the result you are getting is actually a convex hull of hand, not real hand.

Below is my piece of code in OpenCV-Python. I haven't gone for any optimization, just wanted it to work, that's all. If it solves your problem, go for optimization.

import cv2
import numpy as np

def find_if_close(cnt1,cnt2):
    row1,row2 = cnt1.shape[0],cnt2.shape[0]
    for i in xrange(row1):
        for j in xrange(row2):
            dist = np.linalg.norm(cnt1[i]-cnt2[j])
            if abs(dist) < 50 :
                return True
            elif i==row1-1 and j==row2-1:
                return False

img = cv2.imread('dspcnt.jpg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(gray,127,255,0)
contours,hier = cv2.findContours(thresh,cv2.RETR_EXTERNAL,2)

LENGTH = len(contours)
status = np.zeros((LENGTH,1))

for i,cnt1 in enumerate(contours):
    x = i    
    if i != LENGTH-1:
        for j,cnt2 in enumerate(contours[i+1:]):
            x = x+1
            dist = find_if_close(cnt1,cnt2)
            if dist == True:
                val = min(status[i],status[x])
                status[x] = status[i] = val
            else:
                if status[x]==status[i]:
                    status[x] = i+1

unified = []
maximum = int(status.max())+1
for i in xrange(maximum):
    pos = np.where(status==i)[0]
    if pos.size != 0:
        cont = np.vstack(contours[i] for i in pos)
        hull = cv2.convexHull(cont)
        unified.append(hull)

cv2.drawContours(img,unified,-1,(0,255,0),2)
cv2.drawContours(thresh,unified,-1,255,-1)

Below are the results i got :

Answer 2

To fix the connectivity issue, you can try a close operation:

cv::Mat structuringElement = cv::getStructuringElement(cv::MORPH_ELLIPSE, cv::Size(40, 40));
cv::morphologyEx( inputImage, outputImage, cv::MORPH_CLOSE, structuringElement );

I doubt that this will produce the results that you want, but you can give it a try.

Answer 3

It looks like you're "oversegmenting" your image. Morphological operations, as bjnoernz has suggested, would help. In particular, a watershedding approach should get closer to what you want than just checking distance (as in python example above). See http://cmm.ensmp.fr/~beucher/wtshed.html.