OpenCV warpPerspective implementation

Question

I've implemented the least squares method to find the homomorphic image to fix the rotation and projection in an image.

Now I'm trying to implement the OpenCV warpPerspective method in order to "fix" my image, my python implementation is like:

def fix_image(img, t):
    new_image_map = {}
    minx, miny = img.shape[0], img.shape[1]
    maxx, maxy = 0, 0
    for i in range(img.shape[0]):
        for j in range(img.shape[1]):
            xy = np.array([i, j, 1], np.float64)
            uv = np.matmul(t, xy)
            uv = uv / uv[2]
            minx = min(minx, uv[0])
            maxx = max(maxx, uv[0])
            miny = min(miny, uv[1])
            maxy = max(maxy, uv[1])

            new_image_map[int(uv[0]), int(uv[1])] = (i, j)

    minx, miny = int(minx), int(miny)
    maxx, maxy = int(maxx), int(maxy)
    final_img = np.zeros((maxx - minx + 1, maxy - miny + 1)) \
        if len(img.shape) == 2 else np.zeros((maxx - minx + 1, maxy - miny + 1, img.shape[2]))

    for k, v in new_image_map.items():
        final_img[k[0] - minx, k[1] - miny] = img[v]

    return final_img

I know that I still need to interpolate the empty points but the problem is that the shape is not right, I'm checking the results by comparing with the actual OpenCV implementation.

dst = cv2.warpPerspective(storm_img, tr, (1448, 1456))

As you can see, it is far from the expected.

resp = fix_image(storm_img, transformation)

I do not want to just use the OpenCV method because I want to learn how to implement it. What am I getting wrong?

Answer 1

The problem in your implementation is that it returns the tensor of float, while an image must be a tensor of int. Because of that, your rendering library, which I assume is matplotlib, cannot correctly plot an image.

To fix that you need to specify the type of final_img explicitly. That is, you need to add a parameter dtype=np.int when creating final_img, giving us the following:

final_img = np.zeros((maxx - minx + 1, maxy - miny + 1), dtype=np.int) \
        if len(img.shape) == 2 else np.zeros((maxx - minx + 1, maxy - miny + 1, img.shape[2]), dtype=np.int)

You can also refer to this python implementation of cv2.warpPerspective for more details.