2
$\begingroup$

I have two cameras with rotation and translation starting from the world origin and a patch in the first camera's reference coordinate system. I want to warp this patch into the second camera's reference coordinate system. My idea is to transform every pixel like this:

$$T=T_2-T_1$$

$$R=R_1^{T}R_2$$

$$x' = \left[(x K_{inv} R) + T\right]\;K$$

Where $R$ and $T$ are the transformation matrices between the cameras and $K$ is the affine transformation matrix that describes the camera properties.

Is this correct? I know that when you go from 3D to 2D you need to project, but that matrix is not invertible, so you cannot really do it here? Am I missing something really big?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ I don't think you can do what you are trying to do. You would need 3D coordinates for each pixel, and as far as I can tell you have 3D coordinates for each camera but not for the pixels. Getting 3D coordinates for the pixels requires you to know how far each is from the camera, and you can't measure that with just one camera. You would need to identify the same patch on each camera and use the angles and the 3D positions of the cameras to determine the 3D position of each pixel. With that you could warp the 3D pixel coordinates from one camera to the other. $\endgroup$
    – JRE
    Sep 10, 2014 at 10:07
  • $\begingroup$ Unless the patch is flat, you can't perform the mapping, by lack of depth information. Is it flat ? $\endgroup$
    – user7657
    Apr 23, 2018 at 20:22

1 Answer 1

Reset to default
1
$\begingroup$

$x K_{inv}$, with $x(3) = 1$, produces the pixels as 3D points "stuck on the screen" of the first camera. After transforming you get the same 3D points, but in the coordinate frame of the second camera, and then you project in that camera. So, overall, what you are doing is looking at the first camera from the point of view of the second one.

I suspect that is not what you want....

Improve this answer
$\endgroup$
5
  • $\begingroup$ Yeah, that's what I was wondering. I cannot find resources on how warping really works. I think I'm taking it from the wrong angle: I have the 3D points, the patches associated to them and the initial camera where the patches where taken from, maybe I can use the normal of the patch and the rigid transformation together to reproject the patch in the new camera? $\endgroup$
    – aledalgrande
    Jun 11, 2014 at 17:24
  • $\begingroup$ You need to clarify what you mean by "patch": a portion of a plane? A quadrangle on a plane? In either case the transformation between the two images of it is a homography. $\endgroup$
    – Francesco Callari
    Jun 12, 2014 at 3:10
  • $\begingroup$ In this case a patch is a 8x8 pixels feature descriptor. I found the homography from this paper: robots.ox.ac.uk/~lav/Papers/molton_etal_bmvc2004/… but I don't know how to find the normal. Supposedly it is some translation of the Z axis? $\endgroup$
    – aledalgrande
    Jun 12, 2014 at 16:16
  • $\begingroup$ The homography they use is: $K_{1}R[n^{T}x_{p}I-tn^{T}]K^{-1}_{0}u_{0}$ $\endgroup$
    – aledalgrande
    Jun 12, 2014 at 16:28
  • $\begingroup$ Afraid I have no idea of what it means to "warp" a feature descriptor. $\endgroup$
    – Francesco Callari
    Jun 12, 2014 at 23:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.