3
$\begingroup$

Is there any better measurement of error between two images than just taking pixel value differences?

The problem I am tackling is stitching images containing "sensitive" objects, such as moving people. Here are two images I would like to blend:

enter image description here

enter image description here

I compute error map using inter-pixel differences:

enter image description here

The image is then considered a graph where pixels are nodes and edges are connecting neighboring pixels. Edge weights are differences between pixels taken from the error map.

Any Max-Flow/Min-Cut algorithm will find an optimal seam which goes through minimum error path (minimum cut - highlighted in red):

enter image description here

As you can see, the seam have successfully avoided girl's hand, but still goes through the guy's face.

The problem is in the error map which assigns too low error for his face as he just moved his head and the pixel differences of his skin are smaller than ripples on water and other less important features.

I heard about saliency map, but not sure if it is the right approach.

Improve this question
$\endgroup$
4
  • $\begingroup$ What result do you want to get in the blended image? Should there be two guys, or no guy, or maybe one? $\endgroup$
    – Andrey Rubshtein
    Dec 16, 2012 at 14:33
  • $\begingroup$ The ideal result would be having higher error in the face area and hence the seam would go through the right part of the image or between the to people. I don't know about any error measurement that would have inbuilt face detection or "symmetry detection" and I am wondering if there is one. $\endgroup$
    – Libor
    Dec 16, 2012 at 18:25
  • $\begingroup$ I think an optical flow based method would work better here. Than a DSA based method. $\endgroup$
    – Naresh
    Dec 18, 2012 at 5:58
  • $\begingroup$ @Naresh Thanks. I was thinking just about this recently. $\endgroup$
    – Libor
    Dec 18, 2012 at 16:23

1 Answer 1

Reset to default
0
$\begingroup$

I'd mix 2 approaches:

  1. Object Detection
    Train a model to detect the objects you're interested in.
    If you're really sensitive to the location, you may use segmentation.

  2. Saliency Map This will cover areas not covered in the (1) step.
    It will also add a context to the results.

You may use U-Net based methods to do both (Segmentation for (1)).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.