Invariances of FFT-based Image-Registration vs. SIFT-Features

Question

TL;DR: I don't understand how invariant FFT-based image-registration techniques are to object alterations (scratches, marks etc.) in comparison to SIFT-features.

I want to build kind of a feature-matching-like system to detect and compare objects from the same production line. The objects have a granular surface structure, therefore matching surface-features already yields good results.

Here are two sample images of objects I would try to identify:

The extracted features should be invariant against translation, rotation, illumination and partial object-alterations, because one use-case could be failure analysis.

Until now I have realized the matching-process by using SIFT or ORB and matching the descriptors of a query-image against a Database of previously extracted descriptors. If I then iterate over the whole database and count the number of good features (using ratio-test, RANSAC or alike) I can find the matching object by searching for the maximum number of good features.

However this process takes a long time due to the high count of descriptors. That's why I looked into techniques that would generate a hash-string or some comparable vector. I found this paper which seems to accomplish a very similar goal to mine. It's based on this paper, which seems to be kind of reference-paper in the field of image registration. There are some implementations available for the latter, however they always solve the problem of rotated/translated template matching (and here).

Finding out how to do feature extraction with these libraries (as in [1]) would be possible (I guess), but I don't understand whether these processes are also invariant to object-alterations! How would the extracted fft-features look, if I alter the object (scratches, marks, dents etc.)? Would I still be able to match them properly?

Another idea would be to extend the existing feature-matching with Bag of Features to speed up the matching process. Would this be more suitable? Is there another, "better" approach that I haven't considered yet?

Answer 1

I don't understand whether these processes are also invariant to object-alterations!

They are not.

How would the extracted fft-features look, if I alter the object (scratches, marks, dents etc.)? Would I still be able to match them properly?

Yes. But within certain limits.

The Fourier-Mellin transform is based on the Discrete Fourier Transform (DFT) whose coefficients depend directly on its input image's values. Any difference in some $I(x,y)$ will be propagated to $\mathcal{F}(I(x,y))$ and subsequent stages.

In the Fourier Mellin transform, this propagation is even longer because the transform's output is the normalised Discrete Fourier Transform (DFT) of the cartesian-to-polar followed by log transformed amplitude DFT. That is, two DFTs, one on the image and one on circular paths around the amplitude spectrum of the image. But the end result is the same. Changes in $I(x,y)$ propagate to the transform's output. Both transforms are inversible by the way.

In the context of your application, this would introduce some "fuzziness" around the centroid of your classes (For example "Part is good", "Part is defective"), which would be expected. If the texture differences (dents, scratches, etc) are in the very high frequencies it would not even matter for techniques like the one described in the Ishiyama paper where the match is based on part of the spectrum (low frequencies, or, the overall shape of the part).

The extracted features should be invariant against translation, rotation, illumination and partial object-alterations, because one use-case could be failure analysis.

...

Another idea would be to extend the existing feature-matching with Bag of Features to speed up the matching process. Would this be more suitable? Is there another, "better" approach that I haven't considered yet?

It is impossible to achieve all of these goals just at the level of the features.

First of all, what you are dealing with here is probably a One class classifier. You know what "good" looks like and you want to allow parts that are within very close proximity to this estimation of good.

The second point to clarify is the nature of the images. How do defects present themselves on the image? What does a deffective part look like? What does a good part look like? What are the processes that cause these defects? Are you interested in a "yes/no" decision from an image of a part or are you interested in localising where the defect is on the image?

To give you an example of the kind of reasoning you might want to use here: Suppose that we want to quantify how flat a surface that is produced by a planing machine is. We can illuminate it with a source at a very shallow angle that would cause any dents or pits or scratches, to appear as dark streaks or spots on the surface image. Then, simply by thresholding the image and measuring how many "dark pixels" appear in it gives us an estimate of how smooth that surface is. Notice here, there was no reason for fancy features, etc. Only knowledge of the process and the output we are after.

Finally, illumination differences in an industrial environment are easily controlled. And if you have a challenging subject like metalic parts that can produce reflections, you definitely want to control the lighting. You can apply some sort of equalisation or overall normalisation of the brightness, but it is impossible to control for ALL of the artifacts that reflections might produce (epsecially from a single image).

Hope this helps.