Looking for a way to fill holes in objects, and break bridges between them

Question

I am trying to extract building footprints from satellite images. Using a series of morphological operations i.e erosion and opening by reconstruction, I'm able to obtain the footprints, albeit with a lot of noise. Please refer the image below:

As you can see, there are some 'bridges' left between the buildings, and I'm not being able to remove them with opening operations. There are also many holes inside the footprints. I'd love any inputs on how I can improve my result.

P.S I am working on this using openCV and scikit-image on python.

Here's the original image if anyone needs it:

Answer 1

There are two things that make this image particularly difficult to segment using thresholding or a sequence of morphological operations:

• Illumination - some parts of the streets are hit by more sun light than others, hence confused as a foreground (the things you call "bridges").
• Inhomogeneous foreground - the rooftop of a building is not necessarily one color (take for example the most top-left building), hence the "holes" you observe in the footprint.

I managed to get somewhat better results using region-based segmentation and based my solution on this example. Note that I am using SK-image but you should not have problem converting the code to OpenCV as they support the same functions. First, I load and grayscale the image:

img = imread("gfy4e.jpg")
img = rgb2gray(img) * 255

The reason I multiply the grayscaled image by 255 is because it makes the image histogram more readable. Since I used the Watershed algorithm, which requires an elevation map and markers, these are the next things I obtained:

elevation_map = sobel(img)
t = threshold_otsu(img)

markers = np.zeros_like(img)
markers[img < t] = 1
markers[img > t] = 2

Then I apply the Watershed algorithm and fill holes in the resulting image like so:

segmented = watershed(elevation_map, markers)
segmented = binary_fill_holes(segmented - 1)

The noise which remained across the streets I removed by applying an closing operation:

segmented = area_opening(segmented, area_threshold=256, connectivity=1)

You can play around with the hyper-parameters of the functions if the final result does not fully satisfy you. You can also look up Machine/Deep learning approaches towards segmentation as they might be able to give you a better result across wider range of images. Here is the complete code (without the plotting bits):

# Imports:
import numpy as np
from skimage.io import imread
from skimage.color import rgb2gray
from skimage.filters import threshold_otsu, sobel
from skimage.segmentation import watershed
from scipy.ndimage import binary_fill_holes
from skimage.morphology import area_opening

# Solution:
img = imread("gfy4e.jpg")
img = rgb2gray(img) * 255

elevation_map = sobel(img)
t = threshold_otsu(img)

markers = np.zeros_like(img)
markers[img < t] = 1
markers[img > t] = 2

segmented = watershed(elevation_map, markers)
segmented = binary_fill_holes(segmented - 1)

segmented = area_opening(segmented, area_threshold=256, connectivity=1)