Let's say I have two images that are parts of the same bigger image, cut using two masks (for simplicity, let's assume that the masks are just rectangles of the same size). The images are presumed to overlap. The concrete examples are: image stabilization (two different views of the same background) and object tracking (matching the object on two images).
I've read this excellent article http://werner.yellowcouch.org/Papers/subimg/, but it deals with finding the displacement of a small image which is a part of a large one, while I want to find displacement between the images of the same size. My images intersect, but don't contain each other.
I want to find the coefficients of the translation transform which correctly overlaps the matching parts of the images. I want to use something like FFT-based correlation matching. I have the following problems with simply using FFT-correlation:
- The edges of the image frames are strong features. If the algorithm searches the displacement $[dx, dy]$ that maximizes something like $\sum_{x,y}{a[x, y]}b[x + dx, y + dy]$, it would just snap the images so that their frames match, disregarding the content. I heard about windowing, but I don't understand how to apply it here. If each image is tapered at the edges, the images would no longer match when positioned correctly.
- If we use a correlation function that normalizes the result by dividing it by the area of intersection resulting from the displaced matching, the algorithm would just try to keep the intersection to a minimum (~1x1 pixel; the smaller the intersection area is, the easier it is to find a nearly perfect match.)
How to solve this matching problem using windowing or some other techniques?
Update: I know that the problem can be partially eliminated by discarding the low-frequency information (taking gradient or applying the Sobel operator). I'm trying to solve the problem without discarding the low frequency information. Also, leaving only high-frequency features still doesn't solve the outlined problem: Imagine an image with a bright thin horizontal beam and a bleak square. These features are high-frequency and thus will remain after the filter is applied. Cross-correlation will than match the bright beam parts ignoring the square.
Here is the example which shows the matching problem:
Full image with frame borders:
Frames 1 and 2:
Correlation-based incorrect frame matching: