I want to measure the similarity between quantized grayscale images (8 levels) that mostly depict organic patterns found in nature, say pigmentation patterns in animals (zebra pattern, leopard spots,..). I cannot use some kind of trained model, and I need a continuous similarity value. Feature scale tolerance is not important, but differences in rotation and density of the features could be large. Ideally two different flat textures of zebra patterns should have a similarity of 1.0 and say zebra-tiger a similarity of 0.7. I understand how impossible that sounds, but I hope someone could point me to relevant current research or methods somewhat suitable to what I'm looking for.
EDIT: More info about the desired usage.
Examples of images undergone grayscale conversion and quantization: (512x512 pixels, 8 gray levels). I do not own these images.
These are then used as templates for a procedural pattern generation technique I am developing. Images are randomly generated, generation settings that produce high similarity are kept to be further tweaked. The procedurally generated images could be anything from random pixels to intricate shapes, slowly converging at the image of highest possible similarity.