Are there any computer vision algorithms specifically targeting depth images?

Question

I've been looking into marker detection algorithms to use with a kinect based application, and the majority of the work I've been able to find is obviously focused on feature detection in 'normal' images.

However, the kinect hardware provides (essentially, once you've adjusted) an 11-bit depth value per pixel.

This depth image also has various visual artifacts in it from the shadows cast around the edges of objects (see for example the strong black border in this video http://www.youtube.com/watch?v=-q8rRk8Iqww&feature=related).

While some traditional machine vision techniques (eg. edge detection) work well with this, others don't, and it seems like there's little information on the net discussing this.

As a simple example, using the depth value makes it trivial to detect the orientation of a marker block once you've located it.

So, has anyone seen any discussions / papers / etc. that cover processing a depth image for feature detection?

Can anyone recommend a good algorithm for detecting "depth" markers (effectively origami blocks instead of printed b/w markers)?

What I've done so far has been adhoc experimentation using opencv to process the images, but that's no where near stable or fast enough.

If you link to a commercial machine vision product without some kind of trial, please mention in your answer why you think it's appropriate.

Answer 1

My favourite 2.5D/3D feature descriptor for registration and recognition is the spin image (original paper + more details in Ph.D. thesis and software available from CMU).

Other recent advances (all searchable on-line for suitable algorithims) include: 3D-Sift, Fast Point Feature Histogram, Normal Aligned Radial Features (NARF), Depth Kernel Descriptors. Older methods simply used surface properties such as curvature and edges to identify region patches.

Which is best? Depends on what you want to find, viewpoint invariance, additional clutter, etc.

Answer 2

You got all the key words right, I'm surprised that you really didn't find any related articles while looking for material.

Fortunately, I have access to IEEE Xplore digital library. I din't need any of these particular algorithms before, but it looks very interesting so here are some results from a quick search that I think might be relevant (don't judge them by their titles, look at their abstracts):

• Kinezaki, A., Harada, T., Kuniyoshi, Y.: Scale and rotation invariant color features for weakly-supervised object Learning in 3D space
• Bo, L., Ren, X., Fox, D.: Depth kernel descriptors for object recognition
• Ji-Ho Cho, In-Yeop Chang, Seung-man Kim, Lee, K.H.:Depth Image Processing Technique for Representing Human Actors in 3DTV using Single Depth Camera
• Pei-Jun Lee, Effendi: Adaptive edge-oriented depth image smoothing approach for depth image based rendering
• Pedersen, P.C., Quartararo, J.D., Szabo, T.L.: Segmentation of speckle-reduced 3D medical ultrasound images
• Liuhang Cheng, Lisheng Wang, Yu Ma: A new strategy for boundary surface detection in 3D biomedical image

Unfortunately, I don't think you can access any of these papers for free, at least not via the IEEE Xplore library. If you don't have access, you can probably get by with Google scholar, and there are some free paper databases out there (I used Mendeley database back when I didn't have IEEE access yet). Also, just Googling parts of the abstract or random parts of the paper sometimes yields some results (you might stumble across a almost finished pre-published versions of the article).

The search queries I used to find the mentioned papers were: 3D image, depth image, kinect. You might also want to throw in processing when looking up the first two queries.

Hope this helps some! I feel sorry I can not get in to the subject more, sounds really interesting.