# Is this a image similarity problem?

I want to make a programm that can detect the differences between two similar images of an automotive fusebox. The fusebox will be placed inside a jig and so will usually will be in the same position in all the photos taken (won't be rotated, scaled or shifted. There might be subtle differences in lighting throughout the day). Here's an example of a fusebox:

Here's the application: the system will 'learn' a good image and then compare all subsequent fusebox images and check against the good image weather or not this the fusebox was assembled properly. For instance, if a 20A fuse is not placed where it should be, the fusebox is not good.

The fuses themselves are colour coded, blue is always 15A and red is always 10A etc. The fuses we use don't have clear markings on them as shown in the above picture so I can't rely on that, so this leaves character recognition out.

The question now becomes, how do I compare? Should I use histogram based image similarity tests? Would a simple subtraction of the image-under-test from the reference image suffice?

I have developed a very similar system to do this. My algorithm was propriety, but luckily I am also aware of other well performing methods.

First of all, there are many publicly available commercial packages, such as this one. Yet, I would rather be informative on the algorithmic parts on how to implement such a system. According to my experimentation, one good algorithm is provided in the documentation of National Instruments Vision library. You can find it here and here. Their algorithm is also published:

Color characterization for image indexing and machine vision
Siming H. Lin, Dinesh Nair
Proc. SPIE 4116, Advanced Signal Processing Algorithms, Architectures, and Implementations X


You could also use other color similarity metrics. For that, I would refer to the surveys, such as:

Colour difference ∆E - A survey
Mokrzycki W.S., Tatol M.


or

[Similarity and Dissimilarity Measures][4]


Histogram is dependent on the luminosity factor. Since as you mentioned there will be changes in lighting conditions, histogram based methods may not yield the best results.

Try looking at a combination of image segmentation and color characterization.

What you are planning to do sounds like a non-trivial task. Minimizing the variability between the images and the reference image is certainly a way to improve your results - no matter what algorithm you decide for at the end of the day.

My suggestions:

1. Since you are using a jig, why don't you place that jig in a box and use artificial illumination? This way you minimize illumination variances.

2. Place reference colors on the jig. Those can be used to recalibrate the images in case of illumination changes. Besides general colors (e.g. red, green, blue), you can also include all fuses on that reference chart. That way you know exactly what color the fuses are and you can use this information to specifically cut out the corresponding areas in the image via simple histogram manipulations.

3. Generate a binary mask for each fuse type. Apply the mask to the image under investigation and check if the resulting image only consists of the colors that you expect for the fuses that make up that mask image.

4. Use different illuminations (e.g. color filters) and acquire multiple images. Chose the colors in a way that only one type of fuse is bright.