# Hough lines and Convex Hull methods give jagged lines

I need to do a perspective transformation on an image section from newspaper, here the girl's image. The section's coordinates are given as rough estimates. I need to find the boundary of the section and do a perspective transformation. I had been referring the following link.

However, Hough Transform and Convex Hull method on contour finding give jagged lines. This is how the edge detection on the section looks like.

edges = cv2.Canny(gray, 50, 150)


How can I improve the Canny Edge lines to make them straighter so jagged lines are removed? The Convex Hull finds the boundary but not with a number of smaller lines and the boundary is not complete. Here is how the Convex Hull method's boundary looks like.

contours, hierarchy = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
for i, cnt in enumerate(contours):
if hierarchy[0,i,3] == -1 and cv2.contourArea(cnt)>5000:
hull = cv2.convexHull(cnt, returnPoints = True)
print "hull", hull
for tempodd, itemodd in enumerate(hull):
if tempodd % 2 == 0:
tempeven = tempodd
itemeven = itemodd
continue
cv2.line(img, (itemeven[0][0], itemeven[0][1]), (itemodd[0][0], itemodd[0][1]), (255,0,0), 4)
print "itemodd", itemodd, "itemeven", itemeven


Here since the Canny is getting jagged lines, the convex hull method gives multiple lines too. Point me where I am going wrong. Thank you.

• Why do you worry about jagged lines ? If your purpose is to straighten the images, they are completely harmless. What matters is to find accurate edge equations or corner coordinates (actually, accurate transform coefficients). Any global fit method will be immune to the jaggies. I don't think that you are asking the right question.
– user7657
May 26 '15 at 13:35
• I need to get the bounded box out, for which I need a boundary box, and the jagged lines are different lines and on elongating them to get the point of intersection is not working so well, hence I wanted a perfect boundary box around. May 26 '15 at 16:43
• Do you wish to do it automatically? or this is only one time task? Sep 24 '15 at 18:33
• @AlexanderDeLeonVI yes ofcourse automatically, so that all I can do is load the image, and all boxed content are extracted and saved separately. Sep 27 '15 at 3:13

Let's go step by step.

1. First of all you can remove perspective distortion without camera calibration. e.g.

Robust Radial Distortion from a Single Image, Faisal Bukhari and Matthew N. Dailey

Robust Line Based Calibration of Radial Distortion from a Single View, Thorsten Thormahlen, Hellward Broszio, Ingolf Wassermann

2. For any given image, Hough transform might find multiple lines, many of those would be an indicator of the same edge. But luckily you could easily merge the lines which overlap. To detect the overlap you could well use the angle between the lines and the distance of lines. This way, you obtain a clustering of the lines. In any case, finding a short line on each edge would be sufficient for you to recover the projective quad.

3. If you are always sure that the orientation of this quad is more or less fronto parallel, then you could as well use directed gradients (X or Y) to emphasize the vertical or horizontal lines, during the edge detection step.

I agree with the other comments on Gaussian smoothing and anti aliasing. I would not recommend the use of convex hulls though, as they are very noise sensitive.

You could solve the problem in several ways:

1. Image pre-filtering (smoothing) with some kind of Gaussian filter, before doing edge detection, will yield smoother edges.
2. Morphological operations, such as dilate or close on edge image will close the small holes.
3. Find 4 most dominant lines using Hough method on edges image and then find their points of intersection
• I used CV_AA line type argument on cv2.line() function, it removed the jagged lines. Now I am stuck with another problem, distortion in perspective transformations. Thank you @Andrey Apr 30 '14 at 11:06
• you could also try downsampling the image first May 30 '14 at 16:26

So, basically what you want is the bounds of your picture. I think, although doing edge detection will yield corner feature after finding the intersection of the bounding lines, but that is more work.

I would suggest you of finding most dominant four corners by 'corner feature detection'(Using Harris or fast method). Then you have the bounds of your picture. Now you can have the sub image then apply the perspective transform to the raw image or to the corner vertices and then interpolate.

• I tried Shi–Tomasi corner detection algorithm which is entirely based on Harris corner detection method like you said, but it did not get the four corners of the image that i wanted, rather some points in the article like in the holes of "g", "o" letters. I did not delve further into it after that. I will try it again, maybe i did not tweak the parameters well. Apr 30 '14 at 14:42