75
$\begingroup$

As far as I understand, both SURF and SIFT are patent protected.
Are there any alternative methods that can be used in a commercial application freely?

For more info on the patent check out: http://opencv-users.1802565.n2.nabble.com/SURF-protected-by-patent-td3458734.html

Improve this question
$\endgroup$
7
  • 7
    $\begingroup$ Remember they are only patented in countries that allow software patents - which doesn't (yet) include the Eu $\endgroup$
    – Martin Beckett
    Mar 13, 2012 at 15:53
  • 1
    $\begingroup$ @MartinBeckett, does that cover development, deployment, or both? $\endgroup$
    – Andrey Rubshtein
    Mar 13, 2012 at 16:59
  • 2
    $\begingroup$ that's the tricky thing about software patents. A patent stops manufacture or sale in a country but not research or development. Now what is software development? $\endgroup$
    – Martin Beckett
    Mar 13, 2012 at 17:16
  • $\begingroup$ What exactly is patented in SIFT? SIFT has three stages: (i) Construction of scale space, (ii) Keypoint Detector and (iii) Descriptor generator. My feeling is that only the Descriptor Generator is patented. Am I correct? Thanks $\endgroup$
    – user12031
    Dec 10, 2014 at 15:42
  • $\begingroup$ This is not an answer to the question asked, and as such belongs in the comments rather than the answers. $\endgroup$
    – ThP
    Dec 10, 2014 at 15:49

4 Answers 4

Reset to default
73
$\begingroup$

Both SIFT and SURF authors require license fees for usage of their original algorithms.

I have done some research about the situation and here are the possible alternatives:

Keypoint detector:

  • Harris corner detector
  • Harris-Laplace - scale-invariant version of Harris detector (an affine invariant version also exists, presented by Mikolajczyk and Schmidt, and I believe is also patent free).
  • Multi-Scale Oriented Patches (MOPs) - athough it is patented, the detector is basically the multi-scale Harris, so there would be no problems with that (the descriptor is 2D wavelet-transformed image patch)
  • LoG filter - since the patented SIFT uses DoG (Difference of Gaussian) approximation of LoG (Laplacian of Gaussian) to localize interest points in scale, LoG alone can be used in modified, patent-free algorithm, tough the implementation could run a little slower
  • FAST
  • BRISK (includes a descriptor)
  • ORB (includes a descriptor)
  • KAZE - free to use, M-SURF descriptor (modified for KAZE's nonlinear scale space), outperforms both SIFT and SURF
  • A-KAZE - accelerated version of KAZE, free to use, M-LDB descriptor (modified fast binary descriptor)

Keypoint descriptor:

  • Normalized gradient - simple, working solution
  • PCA transformed image patch
  • Wavelet transformed image patch - details are given in MOPs paper, but can be implemented differently to avoid the patent issue (e.g. using different wavelet basis or different indexing scheme)
  • Histogram of oriented gradients
  • GLOH
  • LESH
  • BRISK
  • ORB
  • FREAK
  • LDB

Note that if you assign orientation to the interest point and rotate the image patch accordingly, you get rotational invariance for free. Even Harris corners are rotationally invariant and the descriptor may be made so as well.

Some more complete solution is done in Hugin, because they also struggled to have a patent-free interest point detector.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thanks for your answer. Do they want royalty? $\endgroup$
    – Andrey Rubshtein
    Mar 6, 2012 at 12:55
  • 1
    $\begingroup$ Yes, both of them want royalty fee. The price needs to be negotiated, but it goes around 20.000 USD/year and the royalty fee is about 5%. The MOPs is now patented by Microsoft (I've contacted Richard Szeliski for more info regarding the patent). $\endgroup$
    – Libor
    Mar 7, 2012 at 15:04
  • 1
    $\begingroup$ Patents are public in principle, so if you want to know more about it, look it up in patent databases (e.g. European Database. $\endgroup$
    – Geerten
    Mar 9, 2012 at 12:33
  • $\begingroup$ Are any of those keypoint descriptors scale-invariant? $\endgroup$
    – Diego
    Apr 14, 2012 at 22:25
  • 1
    $\begingroup$ Harris-Laplace is scale-invariant. Or you can make other detectors scale-invariant by filtering out scale space maxima and computing a characteristic scale for each detected point. $\endgroup$
    – Libor
    Jan 31, 2013 at 22:05
28
$\begingroup$

There is a relatively new method, you might want to look into: BRISK, Binary Robust Invariant Scalable Keypoints:

In this paper we propose BRISK, a novel method for keypoint detection, description and matching. A comprehensive evaluation on benchmark datasets reveals BRISK’s adaptive, high quality performance as in state-of-the-art algorithms, albeit at a dramatically lower computational cost (an order of magnitude faster than SURF in cases). The key to speed lies in the application of a novel scale-space FAST-based detector in combination with the assembly of a bit-string descriptor from intensity comparisons retrieved by dedicated sampling of each keypoint neighborhood.

It's patent-free and free to use (as was told by the author of the algorithm).

Improve this answer
$\endgroup$
0
13
$\begingroup$

Don't trust anyone here, talk to a lawyer. The Legal world is subtly different from ours, if I may say. Depending on what you exactly want to do (and where, etc.), there may be a solution where you could use SURF or SIFT. I have been surprised in the past how seemingly strong licenses can be overcome.

Improve this answer
$\endgroup$
9
$\begingroup$

I would rather look into KAZE / AKAZE, which perform equally good with significant speed-up. The deformation cases are also tolerated. OpenCV has recently obtained an implementation through GSoC 2014. You can find it here. Its OpenCV tutorial is also present here.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks. KAZE looks promising - it has better overall performance than SIFT/SURF. Although the nonlinear scale scale computation may be hard to implement, it may worth the effort. $\endgroup$
    – Libor
    Jul 25, 2015 at 1:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.