0
$\begingroup$

I wrote some come to backproject depth maps. Testing it with the Middlebury cone set, I get:

enter image description here

First of all it has noise, which I shouldn't get from the ground truth, but more importantly it seems like the points are going further from each other with the depth.

Can anybody explain this phenomenon?

My code:

  float baseline = 0.3;
  float focalLength = 303.0;
  float numerator = baseline * focalLength;    

  for (int i=10; i<groundTruthDisparity.rows-10; ++i) {
    for (int j=10; j<groundTruthDisparity.cols-10; ++j) {
      float distance = groundTruthDisparity.at<uchar>(i,j) / 4.0;

      if (distance != 0 && !isnan(distance)) {
        depthMap.at<float>(i,j) = numerator / distance;
      }
    }
  }

  for(int y=1; y<height-1; y++) {
    for(int x=1; x<width-1; x++) {
      float depth = depthMap.at<float>(y,x);

      // backproject to world
      cv::Matx21d homogeneous = cv::normalize((cv::Matx21d(x,y) - principalPoint) * (1.0 / focalDistance));
      cv::Matx31d transformedPoint = cv::Matx21d(homogeneous(0), homogeneous(1), 1) * depth;

      cloudPoints.push_back(transformedPoint);
    }
  }

The depth map looks like this (colored in HSV):

enter image description here

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

The reason that pixels get wider with depth is that the depth map works in projection space, so each pixel is basically shaped like a frustum, not like a box. Not sure where the noise is coming from but the depth map doesn't look perfectly clean either.

Improve this answer
$\endgroup$
8
  • $\begingroup$ I changed "wider" to "further from each other". The scanned scene was not cone shaped, so I suppose my reconstruction is wrong? I would expect to see the same width in the point cloud from front to back, don't know if I am explaining it well. $\endgroup$
    – aledalgrande
    Apr 10, 2015 at 23:36
  • $\begingroup$ Did you render with orthographic projection? That'd be the only way that pixels wouldn't get wider (or separate!) With distance. I think the issue is that you need to make your pixels larger as they go into the distance. The exact wideness would depend on your original projection parameters. Your reprojection definitely does not look like orthographic projection, so the issue seems related to this sort of thing (mixing projection modes?) $\endgroup$
    – Alan Wolfe
    Apr 10, 2015 at 23:39
  • $\begingroup$ PS this question will attract a lot of attention on the game Dev stack exchange site if you don't get enough info from here. $\endgroup$
    – Alan Wolfe
    Apr 10, 2015 at 23:42
  • $\begingroup$ To see the point cloud I just use a viewer (Meshlab), which is using a 60 deg projection. I'm still not sure if the problem is in the visualization or in the creation of the depth values. This is the output I would want to have: youtu.be/36TkuvwlfR4?t=47s $\endgroup$
    – aledalgrande
    Apr 10, 2015 at 23:45
  • 1
    $\begingroup$ Oh interesting. My diagnosis was wrong then. I think the noise is just the cracks between the pixels then. Reprojecting depth like this is not my forte at all but I think what you need to do is find a way to weld the corners of the pixels together. A naive solution would be to average the depth of the 4 pixels that meet at a corner, but I bet there are way better techniques that preserve the original structure better! $\endgroup$
    – Alan Wolfe
    Apr 11, 2015 at 0:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.