1
$\begingroup$

My goal is to compare the quality of an image with its low-resolution versions using an object quality metric like PSNR. Since such metrics can only be applied on same-sized images, I first upscale the low-resolution images to the origina size before any metric computation.

However, the images I'm using contain a single object-of-interest in the middle, and the rest of the image is unimportant background. Fortunately, I have the alpha channel stored in the images (sample here) which makes it easy to extract an alpha-mask (sample here) from the image that can be used to distinguish the foreground from the background and calculate the metric only on the foreground pixels (red areas below). The Compare tool of ImageMagick has a nice feature called read-mask which allows doing this in a simple way:

compare -metric PSNR -read-mask mask.png full.png half_upscaled.png diff.png

where full.png is the full-size image (without the alpha channel), mask.png is the alpha mask extracted from the original image, half_upscaled.png is the x2 lower resolution image upscaled to full-size, and diff.png is the generated image after comparison which shows the different pixels between the compared images. An illustration of the image, mask and diff are given below:

enter image description here

I believe it is correct to compute MSE/PSNR on such a masked region because ultimately the computed result depends on the pixel differences between two images. My question would it be conceptually alright to apply another metric like SSIM]6 to such irregular (non-rectangular) regions obtained by masking. My concern is that, since SSIM works on rectangular windows sampled from two images, an irregular pattern like this might lead it to produce false results. Indeed, when I compute PSNR and SSIM between the full and half-res (upscaled to full) images, I get 28.857 and 0.981623, respectively. The SSIM value seems to be quite high although PSNR value is rather medium/low. A second experiment using a quarter-res image (upscaled to full) gave the following result: PSNR: 25.1046, SSIM: 0.957227.

Edit: I understand that the result will strongly depend on the performance of the algorithm used for upscaling the images. This is also another issue to investigate. However, downsampling shouldn't play a role here because in the actual use case, all resolutions are obtained from a 3D engine which renders them in different resolutions, i.e. the original image is not downscaled to obtain low-resolution versions.

$\endgroup$
2
  • $\begingroup$ Did you manage to solve this issue? I have the same question $\endgroup$
    – aCuria
    Jul 15, 2021 at 8:06
  • $\begingroup$ It turned out that this was caused by a bug; see here the Github issue. It seems to be solved by this commit, so if you have a newer version of ImageMagick, you shouldn't have this issue. $\endgroup$ Jul 16, 2021 at 10:23

1 Answer 1

0
$\begingroup$

The SSIM works locally per x, y so you can use only pixels which are within the mask.
You can be more extreme and take only pixels which their neighborhood is within the mask.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.