Can Gabor filter be used for detecting dents in cars?

Question

I am doing some research on Gabor filters for detecting dents in cars. I know Gabor filters have had widespread use for pattern recognition, fingerprint recognition, etc.

I have an image.

Using some code from the MathWorks File Exchange site, I got the following output.

which somehow isn't the output that one would expect. This isn't a good result.

The my script is as follows:

I = imread('dent.jpg');
I = rgb2gray(I);
[G, gabout] = gaborfilter1(I, 2, 4, 16, pi/2);
figure
imshow(uint8(gabout));

EDIT: Applying a different code to the following image :

Output image after different orientations of gabor filter:

how do i isolate this DENT which is got being detected properly?

Answer 1

This is an extremely difficult problem. I was part of a team that worked on it for several years, and having developed and supported other such applications for a long time I can say that dent detection is a particularly tricky problem, and much harder than it looks at first.

Having an algorithm work under lab conditions or on known images is one thing; developing a system that is accurate and robust for "natural" images such as cars seen on a parking lot would likely require a team working for several years. In addition to the core problem of creating the algorithm, there are numerous other engineering difficulties.

The sample code you tested isn't a bad start. If you could find the edges around the dark right side of the dent you could compare the edge map of the car with the ding to an edge map of a known good car imaged from the same angle with the same lighting. Controlling the lighting will help quite a bit.

Problems to consider include the following:

1. Lighting (much more difficult than it would first seem)
2. Expected 3D surface of assembled outer panel (e.g. from CAD data)
3. Criteria characterizing a dent: area, depth, profile, etc.
4. Criteria for false negatives and false positives
5. Means to save dent data and/or map dents onto a model of the car (or butterfly layout)
6. Methodology and device to measure "true" dent characteristics: depth, area, etc.
7. Extensive database of dents from a random sampling of vehicles
8. Dealing with different paint colors and finishes

1. Lighting As Martin B noted correctly above, correct lighting is critical for this problem. Even with good structured lighting, you're going to have great difficulty detecting small dents near feature lines, gaps between panels, handles, and so on.

The Wikipedia entry for structured lighting is a bit thin, but it's a start for understanding the principle: http://en.wikipedia.org/wiki/Structured_light

Light stripes can be used to detect in-dings (dents) and out-dings (pimples). To see a ding, you'll need relative motion between the light source and the car. Either the light + camera move together relative to the car, or the car moves past the light + camera.

Although in-dings and out-dings have characteristic appearances when seen at the edge of a light stripe, the detectability of a given dent depends on the size and depth of the dent relative to the width of the light stripe. A car's curvature is complex, so it's quite difficult to present a consistent light stripe to a camera. As the light stripe moves across the car body, the curvature and even the intensity of the light stripe will vary.

One partial solution is to ensure that the camera and light stripe always at a consistent angle relative to the normal (the 3D perpendicular) of the portion of the surface being inspected. In practice a robot would be required to move the camera accurately relative to the body surface. Moving the robot accurately requires knowledge of the pose (position and 3D angles) of the car body, which is a nasty problem by itself.

For any inspection for automotive applications, you need to completely control the lighting. That means not only placing lights of your choice at known locations, but also blocking all other light. This will mean a fairly large enclosure. Since the car's panels are curved outward (almost like a spherical surface), they'll reflect light from sources all around them. To greatly simplify this problem, you could use a high frequency flourescent bar inside an enclosure shrouded with black velvet. Quite often it's necessary to go to extremes like that for inspection applications.

2. 3D surface A car's outer surface is composed of complex curves. In order to know whether a suspicious spot is a ding, you have to compare that spot to known features of the car. That means you would need to match the 2D image from a camera to a 3D model viewed at a certain angle. This is not a problem solved quickly, and it's difficult enough to do well that some companies specialize in it.

3. Defect characterization For academic research or lab testing it may be sufficient to develop an algorithm that shows promise or improves on an existing method. To properly solve this problem for real commercial or industrial use, you need to have a highly detailed specification for the size dents you'd like to detect.

When we tackled this problem, there were no reasonable industry or national standards for dents (3D deformations). That is, there was no agreed-upon technique to characterize a dent by its area, depth, and shape. We just had samples that industry experts agreed were bad, not too bad, and marginal in terms of severity. Defining the "depth" of a ding is tricky, too, since a ding is a 3D indentation in (typically) a 3D surface curving outward.

Bigger dings are easier to detect, but they're also less common. An experienced auto worker can scan a car body quickly--much more quickly than an untrained observer--and find shallow dings the size of your pinky finger quickly. To justify the cost of an automated system, you would likely have to match an experienced observer's ability.

4. Criteria for detection errors Early on you should set criteria for acceptable false negatives and false positives. Even if you're just studying this problem as an R & D project and don't intend to develop a product, try to define your detection criteria.

false negative: dent present, but not detected false positive: unblemished area identified as a dent

There's typically a tradeoff: increase sensitivity and you'll find more dings (decrease false negatives), but you'll also find more dings that aren't there (increase false positives). It is quite easy to convince oneself that an algorithm performs better than it actually does: our natural bias is to notice defects detected by the algorithm and explain away those it hasn't detected. Conduct blind, automated tests. If possible, have someone else measure the dings and assign severity so that you don't know what the true measurements are.

5. Save data and/or map it A dent is characterized by its severity and its location on the car body. To know its location, you must solve the 2D-to-3D correspondence problem mentioned above.

6. Determining "true" shape of dents Dents are hard to measure. A sharp dent and a rounded dent of the same surface area and depth will appear different. Measuring dents by mechanical means leads to subjective judgments, and it's also quite tedious to use depth gauges, rulers, etc., when you'll likely have to measures dozens if not more.

This is one of the harder engineering problems to solve for any defect detection project for manufacturing: how does one measure a defect and characterize it? if there is a standard for doing so, does the standard correlate well to something the inspection system measures? if the inspection system doesn't find a ding it "should have" found, who's to blame?

That said, if an inspection system works well enough for a sample of known defects, then users may eventually come to trust it, and the system itself becomes the standard for defining defect severity.

7. Extensive database of dents Ideally you would have hundreds if not thousands of sample images of dents of different severities at different locations on vehicles from difficult manufacturers. If you're interested in finding dents caused by accidents during the assembly process, then it could take a long time to collect that kind of data. Dents caused during the assembly process are not common.

If you're only interested in finding dents caused by accidents or environmental damage, then that's a different matter. The types of dents will be different from those caused by accidental bumps inside an auto assembly plant.

8. Dealing with different paint colors It's true that edge detectors can be reasonably robust at detecting edges in images of varying levels of contrast, but it can be quite disheartening to see what "varying levels of contrast" really means for different automotive paints and finishes. A light stripe that looks great on a shiny black car could be hardly detectable on a white car with old paint.

Most cameras have relatively limited dynamic range, so achieving good contrast for both black shiny surfaces and white dull surfaces is tricky. It's quite likely you'll have to automatically control lighting intensity. That's hard, too.

Answer 2

A Gabor filter is only the first of several steps that you would use for such a defect detection task. Also, note that you've only used a specific orientation and scale of Gabor filter -- you would typically use a whole range of orientations and scales.

A very rough outline of a simple defect detection approach using Gabor filters could be as follows:

• Apply Gabor filter at different orientations and scales
• Apply a classifier (trained on suitable training data) to classify each pixel as "defect" or "non-defect"
• Aggregate detections from neighbouring pixels to make sure each defect is detected only once and to eliminate spurious detections

One point that isn't on this list but is typically very important is image acquisition. The way you light your object and place your cameras has a huge influence on how easy it will be to detect defects in the resulting images. Your sample image seems to use the environmental illumination that happened to be present at the time -- this is very likely not optimal.

As I say, this is only a very rough outline. Automatic inspection is a wide field and requires a lot of expertise to do it well.

Answer 3

No, it can't. Image processing is not magic, you need to supply more information. Without some idea of how the car is supposed to look, how is the computer supposed to distinguish a dent from the intended body styling?