8
$\begingroup$

In our unmanned aerial vehicle grayscale video image stabilization application, we're having difficulty finding the "good" Harris corners in frame N+1 selected from frame N. The source of the difficulty appears to be radical nonuniform pixel contrast changes between frames. Perhaps being airborne and using a slow frame rate (~3 fps) is the root cause of the pixel contrast shift.

We've tried various histogram equalization techniques to try to smooth out the pixel contrasts between frames in order to improve tracking "good" Harris corners in frame N+1. Results continue to be poor.

Does anyone have any suggestions on how to improve Harris corner tracking between video frames in a slow frame rate daytime aerial environment? Thank you kindly in advance.

Edit: 30 Jan 2012, added test case (not actual frame size) images

Summary Update: 8 Feb 2012. People suggest Harris corners are not so useful in grayscale video feature tracking. Answers below suggest and provide links to various alternatives. We are evaluating these alternatives and I will report results when we get to that point. Thank you, all, for your comments and answers.

Here's the previous frame N with 35 "good" 5x5 harris corners selected. The original frame is 8 bpp raw pixels.

previous frame N

Here's an excellent 5x5 Harris corner located at row 59 col 266:

previous 5x5 at r59 c266

The current frame N+1 with a few tracked 5x5 Harris corners, only one of which is valid:

current frame N+1

The previous frame 5x5 Harris corner appearing in frame N+1 at r47 c145:

previous 5x5 at current 5x5 r47 c145

Note how the pixel intensities in the selected 5x5 have all changed in a nonuniform manner from the previous frame to the current frame. Contrast equalization techniques between frames do not help in detecting previous frame selected 5x5 pixels in the current frame. All suggestions welcome.

Improve this question
$\endgroup$
3
  • 3
    $\begingroup$ Pl upload a few images. Else you will get random suggestions. $\endgroup$
    – nav
    Commented Jan 27, 2012 at 16:53
  • $\begingroup$ Will do. Sorry, I should have thought of that. $\endgroup$
    – David Pointer
    Commented Jan 27, 2012 at 16:58
  • $\begingroup$ @nav Done! Thank you for your excellent suggestion. $\endgroup$
    – David Pointer
    Commented Jan 30, 2012 at 17:06

4 Answers 4

Reset to default
9
$\begingroup$

Can you try a different feature detector? FAST may be, erm, faster, and a higher frame rate will make matching easier (assuming your features are moving a lot between frames)

Looks like you are trying to use the grayscale region around the identified feature point to match from frame to frame. This is likely to be poor, especially if there is lots of movement between frames.

You may get better performance by using what is often called a "descriptor" of the region around the feature point on which to match.

Some possible descriptors are used within the SURF and SIFT algorithms - they also have techniques for identifying regions to track, but you've bypassed the requirement for that. A simpler descriptor designed for use in stereo-vision matching is the Census transform although that may not work as well for matching from frame to frame, for much the same reason as your current method (which is also widely deployed in stereo-vision)

The book to read on this is Multiple View Geometry in Computer Vision.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for your suggestion. The fps limiting factor is actually the camera in the system - very large frame sizes. We could actually go up to 12 fps with these frame sizes with the current set of algorithms with the technology we're using. $\endgroup$
    – David Pointer
    Commented Jan 30, 2012 at 17:13
  • $\begingroup$ Ohhhh, wait. Are you saying a higher camera frame rate avoids this intensity/contrast shifting problem altogether since there is less time available for the intensities to actually change on observed objects between frames? The airframe itself can move a lot in 33 milliseconds. $\endgroup$
    – David Pointer
    Commented Jan 30, 2012 at 18:13
3
$\begingroup$

As alternative to SIFT/SURF/Other you can also use FFT phase correlation, if frames transformed by mostly translations (rotation/perspective is small). You can also apply phase correlation to regions of image iteratively for better precision.

http://en.wikipedia.org/wiki/Phase_correlation

Improve this answer
$\endgroup$
1
$\begingroup$

If you're trying to align the two images, you should use a better local feature detector. SIFT is probably the most popular/successful one to use.

Improve this answer
$\endgroup$
1
$\begingroup$

I think that is better to use Shi and Tomasi, you can use them with the same function goodfeaturestotrack, it gave better results than the harris corners

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.