Understanding Depth Map / Depth Image / Disparity Map / Disparity Image

Question

Please help me understand the difference between a depth map, a depth image, a disparity map, and a disparity image as I am very confused about the relation between them and how each is used in practice.

After reading things online, my current understanding is that a depth map holds actual distance values in say meters at each pixel location while the depth image holds at each pixel location, a pixel intensity representation of that distance. Is this true?

For a 16-bit uint depth image for example if the depth image holds at position (i, j) an intensity value of 65535, the corresponding depth map will have at position (i, j) a distance of 65535/512 = 128 meters. Is this true?

If what I stated above is true, this would suggest that one unit of pixel intensity in the depth image represents 1/512 meters and I was wondering why is it that one unit of the pixel represents 1/512 meters and not say 1/1024 meters of even 1/128 meters for example. How do we determine that conversion factor.

when we say that depth = (baseline * focal length) / disparity), do we mean that:

depth_image = (baseline * focal length) / disparity_image) in the pixel intensity domain

or that do we mean instead that:

depth_map = (baseline * focal length) / disparity_map) in the distance in meters domain.

That is because the proportionality factor k = (baseline * focal length)will be in a unit of square meters m^2 which will require us to always do such conversion in the distance (meters) domain so that the equation is balanced.

If we were to do this in the pixel domain instead and write the equation as:

depth_image * disparity_image = baseline * focal length

we will have px^2 = m^2 which is not balanced metric wise pending some factor to either get px^2 = px^2 or m^2 = m^2.

In research papers, when deep learning methods are used to predict depth or disparity, are they directly predicting depth images and disparity images i.e. the pixel intensity values? or are they regressing the actual distances in meters values and then converting it back to pixel values for visualization.

If I have a depth image uint 16-bit from some dataset with not other information, is it always true that because I able to read in that depth image and display it, it must be the pixel intensity values of the depth that such dataset is providing.

If I wanted to go from such depth image from the dataset to a disparity image, would I just need to apply disparity_image = (baseline * focal length) / depth_image in the pixel intensity domain or do I need to first convert the given depth image to distances (meters), then apply the formula, then convert the obtained disparity map (in meters) to pixel value before I can use it to train any models.

Thank you very much for you help.

Answer 1

I understand the confusion as these terms are related but have distinct meanings. Let me clarify the difference between a depth map, a depth image, a disparity map, and a disparity image:

Depth Map: A depth map is a representation of the distance or depth information for each pixel in a scene. It is typically a two-dimensional array where each pixel contains a value that represents the distance from the camera to the corresponding point in the scene. The depth values are usually represented in metric units, such as meters. A depth map provides a continuous representation of the scene's depth information.

Depth Image: A depth image is a visual representation of the depth map. It is similar to a regular image, but instead of displaying colors or intensity values, it visualizes the depth values as grayscale or color-coded pixels. Darker regions in the depth image represent objects that are farther away, while brighter regions represent objects that are closer to the camera. The depth image provides a way to visualize the depth information in a scene.

Disparity Map: A disparity map is a type of depth map that is commonly used in stereo vision and 3D reconstruction. It is computed by comparing the disparities between corresponding pixels in two or more images of the same scene, taken from different viewpoints (usually from two cameras). Disparity refers to the horizontal shift of pixels between the images, which is caused by the parallax effect. The disparity map contains pixel values that represent the horizontal shift (disparity) between the images and can be converted into a depth map using known camera parameters.

Disparity Image: A disparity image is a visual representation of the disparity map. Similar to the depth image, it is a visual representation of the computed disparities between the images. Darker regions in the disparity image indicate larger disparities (more distant objects), while brighter regions represent smaller disparities (closer objects).