In a camera-projector system, if only the camera's intrinsics are known, is it possible to generate the depthmap fully automatically through a graycode sequence without any manual calibration process (like using a printed calibration pattern then move it away etc.) ?


1 Answer 1


... if the camera's intrinsics are known and calibrated, is it possible to generate the depthmap fully automatically through a graycode sequence without any manual calibration process ?

Yes, provided that you are talking about a stereo-vision system.

Simply extracting the depth information, does not require knowledge of the camera model. It only requires knowledge of the pixel positions that refer to the same point in space and this is what the Gray code pattern does.

Every Gray code pattern that is projected on to the image "tags" a certain point in space. It is also useful to think about Gray code patterns as bit planes. During the "encoding" process, you segment the image pixels and assign them to bit planes. In this way, every point in space is addressed by a unique binary number that is formed by stacking the bit planes.

If you repeat this process for the left and right images, you will obtain two pairs of $x,y$ coordinates that will tell you where a particular point in space appears to be from the viewpoint of each camera. This is enough to work out "depth" by triangulation.

Knowing the intrinsic parameters allows you to relate this "pixel" depth map to physical coordinates.

For more information, please see this (or any other relevant introduction to imaging using structured light) and this.

Hope this helps.


The basic formula for recovering depth is $Z = f \frac{D}{d_p}$ where:

  • $f$ is the focal distance of the camera lenses.
  • $D$ is the distance between the cameras providing the stereo vision
  • $d_p$ is the "disparity" or relative difference of the position of two pixels associated with the same region in the two images.
  • $Z$ is the distance from the baseline plane which in turn depends on the relative placement of the two cameras.

Now, if the cameras are not fully calibrated, it is impossible to relate $Z$ with reality as the relative distances would not be preserved. BUT, in an extreme case, you can set $D=1$ and obtain a depth map that gives you some information about the depth of the scene. This should "work" for reasonable assumptions of rectilinear lens and aligned cameras and probably return enough information to discriminate between a ball and a cube, if that is the interest of the application here (?)

  • $\begingroup$ Can you clarify more? Without knowing the extrinsics and geometric relationships between the camera and the projector, how do I triangulate the depth map from correspondence map? $\endgroup$
    – Oct F
    Oct 26, 2018 at 13:31
  • $\begingroup$ @OctF How do you not know the extrinsics? You must at least know the focal distance of the lenses (which you have) and the distance between the cameras to solve for distance / depth assuming that they have a parallel orientation (see the uniroma page). Are you working on a specific problem or are you trying to recover data from a given dataset? $\endgroup$
    – A_A
    Oct 26, 2018 at 15:03
  • $\begingroup$ as the question states,extrinsics and camera distance are unknown.because there is no manual calibration involved.All I want is to study if there's possibility to do what described in the question. $\endgroup$
    – Oct F
    Oct 26, 2018 at 16:43
  • $\begingroup$ @OctF I have provided some additional information having reviewed the edits to your original question and these comments. It would help knowing what sort of application you are dealing with. Strictly speaking, if you don't know how the cameras are related to each other, it is impossible to derive depth. Some of these parameters can be supplied manually by the user as opposed to being derived. $\endgroup$
    – A_A
    Oct 27, 2018 at 9:08
  • $\begingroup$ many thanks, i'd like to offer a thumb up for your kind effort, but I cannot accept this as the solution for now. $\endgroup$
    – Oct F
    Oct 28, 2018 at 9:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.