# How to apply a homography matrix after finding it via a common camera calibration technique

I am trying to find a matrix that maps four specific 2D-points to another set of specific 2D-points.

Specifically, I'm trying to find the matrix $H$, such that $x_i=HX_i$ achieves the following mapping: $$X_1 = \begin{bmatrix}0 \\ 0\end{bmatrix} \mapsto x_1 = \begin{bmatrix}0 \\ 10\end{bmatrix}$$

$$X_2 = \begin{bmatrix}2 \\ 0\end{bmatrix} \mapsto x_2 = \begin{bmatrix}2 \\ 9\end{bmatrix}$$

$$X_3 = \begin{bmatrix}0 \\ 4\end{bmatrix} \mapsto x_3 = \begin{bmatrix}1 \\ 14\end{bmatrix}$$

$$X_4 = \begin{bmatrix}2 \\ 4\end{bmatrix} \mapsto x_4 = \begin{bmatrix}4 \\ 11\end{bmatrix}$$

Here is what I've done so far.

Set up a homographic mapping in homogeneous coordinates: $$\left[\begin{matrix}wx_i\\wy_i\\w\\\end{matrix}\right]=\left[\begin{matrix}h_{11}&h_{12}&h_{13}\\h_{21}&h_{22}&h_{23}\\h_{31}&h_{32}&h_{33}\\\end{matrix}\right]\left[\begin{matrix}X_i\\Y_i\\1\\\end{matrix}\right]$$

Write it as follows: $$\begin{matrix}wx_i=h_{11}X_i+h_{12}Y_i+h_{13}\\wy_i=h_{21}X_i+h_{22}Y_i+h_{23}\\w=h_{31}X_i+h_{32}Y_i+h_{33}\\\end{matrix}$$

Plug in $w$ for the first two equations: $$\begin{matrix}\left(h_{31}X_i+h_{32}Y_i+h_{33}\right)x_i=h_{11}X_i+h_{12}Y_i+h_{13}\ \\\left(h_{31}X_i+h_{32}Y_i+h_{33}\right)y_i=h_{21}X_i+h_{22}Y_i+h_{23}\\\end{matrix}$$

Solve to get in a homogeneous form: $$\begin{matrix}h_{31}X_ix_i+h_{32}Y_ix_i+h_{33}x_i-h_{11}X_i-h_{12}Y_i-h_{13}=0\ \\h_{31}X_iy_i+h_{32}Y_iy_i+h_{33}y_i-h_{21}X_i-h_{22}Y_i-h_{23}=0\\\end{matrix}$$

Write in matrix form: $$\left[\begin{matrix}-X_i&-Y_i&-1&0&0&0&X_ix_i&Y_ix_i&x_i\\0&0&0&-X_i&-Y_i&-1&X_iy_i&Y_iy_i&y_i\\\end{matrix}\right] \begin{bmatrix} h_{11}\\ h_{12} \\ h_{13} \\ h_{21} \\ h_{22} \\ h_{23} \\h_{31} \\ h_{32} \\ h_{33}\end{bmatrix}=\begin{bmatrix}0 \\ 0\end{bmatrix}$$

Set up the matrix for four points: $$\left[\begin{matrix}\begin{matrix}-X_1&-Y_1&-1&0&0&0&X_1x_1&Y_1x_1&x_1\\0&0&0&-X_1&-Y_1&-1&X_1y_1&Y_1y_1&y_1\\\end{matrix}\\\begin{matrix}-X_2&-Y_2&-1&0&0&0&X_2x_2&Y_2x_2&x_2\\0&0&0&-X_2&-Y_2&-1&X_2y_2&Y_2y_2&y_2\\\end{matrix}\\\begin{matrix}-X_3&-Y_3&-1&0&0&0&X_3x_3&Y_3x_3&x_3\\0&0&0&-X_3&-Y_3&-1&X_3y_3&Y_3y_3&y_3\\\end{matrix}\\\begin{matrix}-X_4&-Y_4&-1&0&0&0&X_4x_4&Y_4x_4&x_4\\0&0&0&-X_4&-Y_4&-1&X_4y_4&Y_4y_4&y_4\\\end{matrix}\\\end{matrix}\right] \begin{bmatrix} h_{11}\\ h_{12} \\ h_{13} \\ h_{21} \\ h_{22} \\ h_{23} \\h_{31} \\ h_{32} \\ h_{33}\end{bmatrix}=\begin{bmatrix}0\\0\\0\\0\\0\\0\\0\\0\end{bmatrix}$$

My objective now is to solve for $h$ in the following equation: $$Ah=0$$

In other words, I want to find a basis for the nullspace of $A$.

Since the nullspace is associated with the singular values that are zero, I compute the SVD of $A$, which has a single zero as a singular value which correpsonds to the right most column vector of $V^{T}$ in the following equation: $$\textrm{svd}(A) = UDV^{T}$$

This right most column vector of $V^{T}$ is the solution of $Ah=0$.

I then reshape $h$ into a $3\times3$ matrix: $$H=\textrm{reshape(h,3,3)}= \left[\begin{matrix}h_{11}&h_{12}&h_{13}\\h_{21}&h_{22}&h_{23}\\h_{31}&h_{32}&h_{33}\\\end{matrix}\right]$$

In MATLAB I do this as follows:

X = [
0,0;
2,0;
2,4
0,4;];

x = [
0,10;
2, 9;
4,11
1,14;];

X1 = X(1,1);    Y1 = Y(1,2);
X2 = X(2,1);    Y2 = Y(2,2);
X3 = X(3,1);    Y3 = Y(3,2);
X4 = X(4,1);    Y4 = Y(4,2);

x1 = x(1,1);   y1 = x(1,2);
x2 = x(2,1);   y2 = x(2,2);
x3 = x(3,1);   y3 = x(3,2);
x4 = x(4,1);   y4 = x(4,2);

A = [
-X1, -Y1, -1,   0,   0,  0, X1*x1, Y1*x1, x1;
0,   0,  0, -X1, -Y1, -1, X1*y1, Y1*y1, y1;
-X2, -Y2, -1,   0,   0,  0, X2*x2, Y2*x2, x2;
0,   0,  0, -X2, -Y2, -1, X2*y2, Y2*y2, y2;
-X3, -Y3, -1,   0,   0,  0, X3*x3, Y3*x3, x3;
0,   0,  0, -X3, -Y3, -1, X3*y3, Y3*y3, y3;
-X4, -Y4, -1,   0,   0,  0, X4*x4, Y4*x4, x4;
0,   0,  0, -X4, -Y4, -1, X4*y4, Y4*y4, y4
];

[U,S,Vt] = svd(A);

v9 = Vt(:,9);

% Test to ensure that v9 is actually basis of nullspace
does_map_to_zero_vector = A * v9;

H = reshape(v9,3,3);


I'm pretty sure this is correct because when I pass those four corresponding points to MATLABs built in estimateGeometricTransform() function I get the same matrix (within a scale factor). The result (after scaling), is:

H * 10.17655 =
1.2500    0.6250    0.1250
0.1250   -0.7500   -0.1250
0        10.0000    1.0000


This is where I am confused. How do I apply this matrix $H$ to achieve that desired mapping of the four points?

Do I simply place a $1$ in the the 3rd-element of each vector to get a homogeneous form and then multiply by $H$ and then divide each element by the 3rd-element to get out of homogeneous form as follows?

Converting to homogeneous coordinates before applying transformation: $$\begin{bmatrix} x \\ y \end{bmatrix} \mapsto \begin{bmatrix} x \\ y \\ 1 \end{bmatrix}$$

Converting from homogeneous coordinates after applying transformation: $$\begin{bmatrix} x' \\ y' \\ w \end{bmatrix} \mapsto \begin{bmatrix} \frac{x'}{w} \\ \frac{y'}{w} \end{bmatrix}$$

When I apply my last few equations I don't get the correct mapping. For example: $$\begin{bmatrix} 2 \\ 0 \end{bmatrix} \mapsto \begin{bmatrix} 2 \\ 0 \\ 1 \end{bmatrix}$$

$$\begin{bmatrix} 1.25 & 0.625 & 0.125 \\ 0.125 & -0.75 & -0.1250 \\ 0 & 10.0000 & 1.0000 \end{bmatrix} \begin{bmatrix} 2 \\ 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 2.625 \\ .0125 \\ 1 \end{bmatrix}$$

Which converts back to non-homogeneous coordinates by dividing the first two elements by 1, which is $\begin{bmatrix}2.625 \\.0125\end{bmatrix}$.

But it should map to $\begin{bmatrix}2 \\9\end{bmatrix}$

What am I doing wrong?