I have a problem in which i have a video that was taken from the side of the subject, something like this enter image description here and i need to transform the coordinates of the subject to be as if the photo was taken from above like this enter image description here

I have the real set of coordinates that i can relate to from picture 2 to picture 1 , i only need to transform the coordinates of the moving subject, not the whole picture.

I tried using polynomial regression in order to approximate the right transformation, but it failed to work (i can see that it's not the true movement).

also i have a problem that the video is taken with a fish eye lens.

what is the right way to approach it? or if you can direct me to the appropriate papers? (i never done image processing before).

many thanks,

  • $\begingroup$ Do you mean that you have the actual mapping from 1 to 2? I can not comment so I pose this here. There are transforms of various nature. They depend on the data you have. For example - do you know the camera location? Do you know the actual sizes of the squares in one? $\endgroup$
    – Moti
    Mar 9, 2015 at 17:38
  • $\begingroup$ im not sure i fully understand what do you mean by mapping. I have all the measurements of the room, i know the size of the grid and also i know where the camera is. from what i read i understood that a projection transformation might do, but i'm not sure how well it will handle the fish eye distortion. thanks. $\endgroup$
    – Kozolovska
    Mar 10, 2015 at 7:58
  • $\begingroup$ There are simple 2D transforms for tilting rotating and zooming; but the fisheye is a problem. I could imagine the transform being slightly position dependent if you actually could use the corners as alignment "crosses"; but you keep mentioning "motion". What do you mean? $\endgroup$
    – rrogers
    Mar 10, 2015 at 21:35
  • $\begingroup$ Well basicly, there is a room with a camera and it films movement of an subject, after that there is a software that extract his coordinates (X,Y in the image plane). i need to transform those coordinates to be as if the video was taken from above the subject. what do you mean by crosses as alignment? can you direct me to a paper or an algorithm? thanks $\endgroup$
    – Kozolovska
    Mar 11, 2015 at 7:28

2 Answers 2


I can't help with the fisheye lens issue, but assuming that effect isn't too large, the mapping between the above images is a perspective transformation.

We can transform a point on the plane $z=1$ into 3D space using a $3\times 3$ matrix.

$$\left( \begin{array}{c} X\\ Y\\ Z\\ \end{array} \right) = \left( \begin{array}{c} s_x\ \ h_x\ \ t_x \\ h_y\ \ s_y\ \ t_y \\ p_x\ \ p_y\ \ 1 \\ \end{array} \right) \left( \begin{array}{c} x\\ y\\ 1\\ \end{array} \right)$$

  • $s$ is scaling
  • $h$ is shearing
  • $t$ is translation
  • $p$ is perspective tilting (positive number means positive side tilts away)

We can project this object in 3D space back onto the plane $z=1$ simply by dividing the coordinates by $Z$. This maps every point in space to the intersection of the plane $z=1$ and the line connecting the point and the origin. This is basically the process of taking a photograph.

$$X_{proj} = X/Z$$ $$Y_{proj} = Y/Z$$

What we want to do is find the reverse transformation that takes us from the projected coordinates back to the original coordinates where the points only existed in the plane $z=1$.

It turns out that we can solve for all of the transformation parameters using 4 points (the four corners of a rectangle, for example) and solving the following system of equations (see here for the derivation and some example MATLAB code).

enter image description here

Making certain assumptions like the object not changing size or shape ($s_x=s_y=1$, $h_x=h_y=0$) can simplify these equations.

  • $\begingroup$ Hi, Thanks very much for the help. i have a few questions still, i have lets say the P coordinates from the image. i want to transform them to the P' coordinates of the plane i want to project them. first, i don't have the real coordinates of the plane i have them in meters and i don't think they correspond directly to the P coordinates. and also does i mean that every coordinate i want to project to i need the multiply it by H i mean P*H = P' ? $\endgroup$
    – Kozolovska
    Mar 12, 2015 at 11:26

The following link from math.stackexchange may be of further help to you, if only to prove you're not alone...


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.