1
$\begingroup$

I am trying to calculate the disparity between stereo image pairs. I'm using what appears to be the standard approach of correlating between a window of pixels from one image and a sliding window from the other. The horizontal shift that gives the best correlation is the disparity.

However, the images contain large almost featureless areas. In those areas, the calculated disparity is frequently wrong. I would like to introduce a correlation confidence measure, so that the results in these areas can be discarded.

Is there a common measure of confidence that is suitable for this application? At the moment I am struggling to distinguish between truly featureless areas, and those with low contrast that still have recognisable features.

$\endgroup$
1
$\begingroup$

First, on distinguishing low contrast features from noise, then on a possible alternate course of action to solve what I think you are trying to do.

From my experience, distinguishing between noise and low contrast features is a hard thing to do. The problem arises quite often when you try to do edge detection on an image. Using the most common algorithms such as Canny, Sobel, LoG, etc. generally results in a good deal of noise, which is dependant of the parameters you set (e.g.: a 'threshold' needs to be set (chosen) in order to define at which intensity value an edge-candidate pixel is classified as part of the edge or not, so what if the noise is more 'intense' than the correct edge?)

So, what's the point with stereo correspondence/disparity? Some scholars tackled on the problem of noisy edge detection and there is right now a pretty good edge detector, namely the Berkeley edge detector, which use machine learning techniques to 'improve' the edge detection.

Here are 3 images, left is original, centre is with Canny edge and right is with Berkeley. (Images credit: ETHZ-Shape-Class Dataset)

ETHZ-Shape-ClassesETHZ-Shape-Classes-CannyETHZ-Shape-Classes-Berkeley

You can see that (while maybe some tuning could have been done on Canny) Berkeley is providing a 'better' (less noisy) edgemap than Canny.

So, you could see that there are techniques that you could possibly try to adapt in order to tell the salient features from the noise. I can't guaranty it will work since I haven't done it myself. Also, please note that these machine learning algorithms are pretty slow which prevent their use in realtime processing of images.


This said, what you really want to do is measure the dissimilarity between the to images by mean of correlation techniques, which was not successful apparently. Let me tell you about what is known as keypoint-descriptors. You could read on this topic there. The broad idea is to be able to evaluate the correspondence of some (a lot of) features between two images. It is often used in object detection (where you want to find the presence and location of a certain object in an input image) or in image stitching (such as panoramas maker, where you want to 'align' two or more images). Now, how can you tell that you have two images corresponding to the same object? These methods using keypoint-descriptors are, in the end, giving a measure of similarity between both images. Actually, lots of them are measuring the dissimilarity between the features to tell if they are similar or not...

I hope you will be able to better understand the situation with this. This is not a direct pre made solution, but it could help you in your work!

$\endgroup$
  • $\begingroup$ If you need precision on parts of this, let me know! $\endgroup$ – Doombot Sep 25 '14 at 14:12
  • $\begingroup$ Good answer. Could you provide some references to make it excellent? $\endgroup$ – Andrey Rubshtein Jun 22 '15 at 22:56
  • $\begingroup$ As a general reference, I generally use Szeliski book (here is a draft szeliski.org/Book). For the edge detection part, here is Ferrari's paper scholar.google.ca/… $\endgroup$ – Doombot Jun 25 '15 at 19:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.