OpenCV: Can't find large rectangle contour

Question

I'm trying to re-create the dimensions of an object by setting it up on a grid and taking as close to a top-down photo I can which I will then get the contours of the largest bounding rectangle and then perspective warp.

I'm currently unable to get the contour for a large bounding square however, it continually only finds smaller rectangles/squares which I'm assuming would not be large enough to properly fix the perspective.

First image: Original

Second image: What I get with my code using openCV

Third image: Close to what I'd ideally get

My code:

import imutils
import numpy as np
import cv2 as cv

# load the query image
image = cv.imread("path/to/image")

# make image greyscale, blur, find edges
grayscale_image = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
thresh = cv.adaptiveThreshold(grayscale_image, 255, cv.ADAPTIVE_THRESH_GAUSSIAN_C,
                              cv.THRESH_BINARY, 11, 2)

# find contours in the threshed image, keep only the largest
# ones
cnts = cv.findContours(
    thresh.copy(), cv.RETR_LIST, cv.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts)
cnts = sorted(cnts, key=cv.contourArea, reverse=True)[:5]

# draw contours for reference
cv.drawContours(image, cnts, -1, (0, 255, 0), 3)

Instead of adaptive thresholding for pre-processing I've tried using bilateral filter or gaussian blur into canny edge detection but the outcome still doesn't find large rectangles.

Any help would be greatly appreciated as I'm at a loss on why it can't detect larger squares. Also, if people think there's a better method for fixing the perspective so that I can accurately recreate the board dimensions please let me know.

Answer 1

My guess is you can find all rectangles using hough transform. OpenCV python returns a structure that has all rectangles. Then sort the rectangles and find take out the largest one and using the coordinate it give you plot it on the image. Something like the answers in Android OpenCV Find Largest Square or Rectangle.

My suggestion is to use those corner information you have, to register image (maybe using affine model), in case the camera is a bit tilted, skewed.