My goal is to compare the quality of an image with its low-resolution versions using an object quality metric like PSNR. Since such metrics can only be applied on same-sized images, I first upscale the low-resolution images to the origina size before any metric computation.
However, the images I'm using contain a single object-of-interest in the middle, and the rest of the image is unimportant background. Fortunately, I have the alpha channel stored in the images (sample here) which makes it easy to extract an alpha-mask (sample here) from the image that can be used to distinguish the foreground from the background and calculate the metric only on the foreground pixels (red areas below). The Compare tool of ImageMagick has a nice feature called read-mask which allows doing this in a simple way:
compare -metric PSNR -read-mask mask.png full.png half_upscaled.png diff.png
full.png is the full-size image (without the alpha channel),
mask.png is the alpha mask extracted from the original image,
half_upscaled.png is the x2 lower resolution image upscaled to full-size, and
diff.png is the generated image after comparison which shows the different pixels between the compared images. An illustration of the image, mask and diff are given below:
I believe it is correct to compute MSE/PSNR on such a masked region because ultimately the computed result depends on the pixel differences between two images. My question would it be conceptually alright to apply another metric like SSIM]6 to such irregular (non-rectangular) regions obtained by masking. My concern is that, since SSIM works on rectangular windows sampled from two images, an irregular pattern like this might lead it to produce false results. Indeed, when I compute PSNR and SSIM between the full and half-res (upscaled to full) images, I get 28.857 and 0.981623, respectively. The SSIM value seems to be quite high although PSNR value is rather medium/low. A second experiment using a quarter-res image (upscaled to full) gave the following result: PSNR: 25.1046, SSIM: 0.957227.
Edit: I understand that the result will strongly depend on the performance of the algorithm used for upscaling the images. This is also another issue to investigate. However, downsampling shouldn't play a role here because in the actual use case, all resolutions are obtained from a 3D engine which renders them in different resolutions, i.e. the original image is not downscaled to obtain low-resolution versions.