# How to Avoid Perspective Distortion Buildup in a Planar Mosaic

Registering images for map or other planar mosaic from a handheld camera or a drone requires projective transform (homography) as the camera is allowed to rotate between frames. Unfortunately, planar homographies cause mosaic to grow too big with each new image being registered:

Please note I am not trying to make a rotational mosaic, rather a planar mosaic (camera translating) with additional rotation (due to wind, hand rotation etc.).

To avoid this issue, I have allowed homography only for image-to-image registration, but the images are then globally aligned using simple translation, that minimizes distance between feature correspondences (originally matched using homography):

As you can see, this produces big misalignments.

I found that Microsoft ICE (Image Composite Editor) offers "Planar Motion with Perspective" option that tackles this problem:

Here is the result from ICE:

I can't figure out how they solved the problem, i.e. how to combine planar mosaic with perspective distortion and avoid the distortion buildup.

I tried splitting homography transform to in-place perspective distortion without translation, then translating the distorted point:

$$\textbf{x'}=\begin{pmatrix}1 & 0 & t_x \\ 0 & 1 & t_y \\ 0 & 0 & 1\end{pmatrix}\begin{pmatrix}h_{11} & h_{12} & 0 \\ h_{21} & h_{22} & 0 \\ h_{31} & h_{32} & 1\end{pmatrix}\textbf{x}$$

Unfortunately, this still produces the distortion buildup.

Another option is to introduce some error term in the bundle adjustment (optimization) routine that discourages too high values of perspective distortions. The problem with extra error test is that it complicates the model and it is hard to find proper scaling for the additional error factor to properly weight other variables.

Yet another solution may be to keep image positions fixed and adjust only perspective distortion. This however is not a globally optimal solution.

Any ideas?