Registering images for map or other planar mosaic from a handheld camera or a drone requires projective transform (homography) as the camera is allowed to rotate between frames. Unfortunately, planar homographies cause mosaic to grow too big with each new image being registered:

enter image description here

Please note I am not trying to make a rotational mosaic, rather a planar mosaic (camera translating) with additional rotation (due to wind, hand rotation etc.).

To avoid this issue, I have allowed homography only for image-to-image registration, but the images are then globally aligned using simple translation, that minimizes distance between feature correspondences (originally matched using homography):

enter image description here

As you can see, this produces big misalignments.

I found that Microsoft ICE (Image Composite Editor) offers "Planar Motion with Perspective" option that tackles this problem:

enter image description here

Here is the result from ICE:

enter image description here

I can't figure out how they solved the problem, i.e. how to combine planar mosaic with perspective distortion and avoid the distortion buildup.

I tried splitting homography transform to in-place perspective distortion without translation, then translating the distorted point:

$$\textbf{x'}=\begin{pmatrix}1 & 0 & t_x \\ 0 & 1 & t_y \\ 0 & 0 & 1\end{pmatrix}\begin{pmatrix}h_{11} & h_{12} & 0 \\ h_{21} & h_{22} & 0 \\ h_{31} & h_{32} & 1\end{pmatrix}\textbf{x}$$

Unfortunately, this still produces the distortion buildup.

Another option is to introduce some error term in the bundle adjustment (optimization) routine that discourages too high values of perspective distortions. The problem with extra error test is that it complicates the model and it is hard to find proper scaling for the additional error factor to properly weight other variables.

Yet another solution may be to keep image positions fixed and adjust only perspective distortion. This however is not a globally optimal solution.

Any ideas?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.