# Transformation of coordinate set

I have a problem in which i have a video that was taken from the side of the subject, something like this and i need to transform the coordinates of the subject to be as if the photo was taken from above like this

I have the real set of coordinates that i can relate to from picture 2 to picture 1 , i only need to transform the coordinates of the moving subject, not the whole picture.

I tried using polynomial regression in order to approximate the right transformation, but it failed to work (i can see that it's not the true movement).

also i have a problem that the video is taken with a fish eye lens.

what is the right way to approach it? or if you can direct me to the appropriate papers? (i never done image processing before).

many thanks,

• Do you mean that you have the actual mapping from 1 to 2? I can not comment so I pose this here. There are transforms of various nature. They depend on the data you have. For example - do you know the camera location? Do you know the actual sizes of the squares in one? – Moti Mar 9 '15 at 17:38
• im not sure i fully understand what do you mean by mapping. I have all the measurements of the room, i know the size of the grid and also i know where the camera is. from what i read i understood that a projection transformation might do, but i'm not sure how well it will handle the fish eye distortion. thanks. – Kozolovska Mar 10 '15 at 7:58
• There are simple 2D transforms for tilting rotating and zooming; but the fisheye is a problem. I could imagine the transform being slightly position dependent if you actually could use the corners as alignment "crosses"; but you keep mentioning "motion". What do you mean? – rrogers Mar 10 '15 at 21:35
• Well basicly, there is a room with a camera and it films movement of an subject, after that there is a software that extract his coordinates (X,Y in the image plane). i need to transform those coordinates to be as if the video was taken from above the subject. what do you mean by crosses as alignment? can you direct me to a paper or an algorithm? thanks – Kozolovska Mar 11 '15 at 7:28

I can't help with the fisheye lens issue, but assuming that effect isn't too large, the mapping between the above images is a perspective transformation.

We can transform a point on the plane $z=1$ into 3D space using a $3\times 3$ matrix.

$$\left( \begin{array}{c} X\\ Y\\ Z\\ \end{array} \right) = \left( \begin{array}{c} s_x\ \ h_x\ \ t_x \\ h_y\ \ s_y\ \ t_y \\ p_x\ \ p_y\ \ 1 \\ \end{array} \right) \left( \begin{array}{c} x\\ y\\ 1\\ \end{array} \right)$$

• $s$ is scaling
• $h$ is shearing
• $t$ is translation
• $p$ is perspective tilting (positive number means positive side tilts away)

We can project this object in 3D space back onto the plane $z=1$ simply by dividing the coordinates by $Z$. This maps every point in space to the intersection of the plane $z=1$ and the line connecting the point and the origin. This is basically the process of taking a photograph.

$$X_{proj} = X/Z$$ $$Y_{proj} = Y/Z$$

What we want to do is find the reverse transformation that takes us from the projected coordinates back to the original coordinates where the points only existed in the plane $z=1$.

It turns out that we can solve for all of the transformation parameters using 4 points (the four corners of a rectangle, for example) and solving the following system of equations (see here for the derivation and some example MATLAB code).

Making certain assumptions like the object not changing size or shape ($s_x=s_y=1$, $h_x=h_y=0$) can simplify these equations.

• Hi, Thanks very much for the help. i have a few questions still, i have lets say the P coordinates from the image. i want to transform them to the P' coordinates of the plane i want to project them. first, i don't have the real coordinates of the plane i have them in meters and i don't think they correspond directly to the P coordinates. and also does i mean that every coordinate i want to project to i need the multiply it by H i mean P*H = P' ? – Kozolovska Mar 12 '15 at 11:26

The following link from math.stackexchange may be of further help to you, if only to prove you're not alone...