# How is downsampling and low-pass filtering done in MS-SSIM?

I am just studying the Multiscale Strutural Similarity measure for image quality and going through the original paper [1].

I think I understood the basic ides and "regular" structural similarity, but could someone clarify on how downsampling is done and which low-pass filter is applied when computing the MS-SSIM?

[1] Wang, Zhou, Eero P. Simoncelli, and Alan C. Bovik. "Multiscale structural similarity for image quality assessment." The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003. Vol. 2. Ieee, 2003. Available online.

I assume the question is specifically the decimate by 2 as in this figure:

Decimation is the combination of a low pass filter with a downsampler. The downsampling by 2 designated by the block that has a 2 with a down-arrow is simply done by removing every other sample. This would fold in frequency everything in the upper part of the spectrum so an anti-alias filter is required prior to the down-sampling. This is no different from the requirement for an anti-alias filter prior to an A/D converter. So the ideal filter L would be a low pass filter that would pass everything that would be in the first Nyquist zone defined by the new sampling rate after the downsamping, while completely rejecting all the high frequency components that would otherwise fold in. (Such an ideal filter cannot be realized, but we approach this trading complexity with allowable distortion).

This graphic I have below demonstrates this with a decimate by 4 for a real signal with the first Nyquist zone extending from $$0$$ to $$F_s/2$$, where $$F_s$$ is the sampling rate (and $$-F_s/2$$ to $$+F_s/2$$ for complex signals).

With digital sampling (or resampling as in this case) all the spectrum centered on $$mF_s$$ is aliased to be centered on $$0$$ in the first Nyquist zone where $$m$$ is any integer, so here in this example the new sampling rate is $$1/4$$ of what the original sampling rate, thus we see all the images that are at multiples of this new sampling rate that were part of the original digital spectrum that was at the higher rate.

In this case a multi-band rejection filter is a good solution to minimize resources. Alternate approaches that are also quite efficient are Cascade-Integrator-Comb structures (CIC) and polyphase filters which are detailed in other posts on this site.

• Thanks for the quick an detailed reply! But as I'm just a beginner in the field, yet, I'm not sure how to understand your answer. Is the following correct? We downsample by a factor of two means we use a sampling rate which is half of the original one, i.e. we discard every second sample. All frequencies higher than the new nyquist frequency can cause aliasing, therefore we filter them out by applying an appropriate low pass filter before downsampling. Correct so far? Apr 13 '20 at 14:58
• Yes you got it, and to see aliasing as a beginner, consider looking at a spinning bicycle wheel with a strobe light--- If the wheel was spinning 12 cycles/sec and we strobe it at 10 cycles/sec, the wheel would appear to be spinning at 2 cycles/sec. Thus whether the wheel was spinning at 12 cycles or 2 cycles both would appear the same once we sample. This is aliasing Apr 13 '20 at 15:00
• Very nice analogy! Thanks! Can I bother you with a follow-up question? I can image this now for a "1D signal" but how does it apply to (greyscale) images? Can we implement downsampling just by any kind of pooling? And this implementation appears to me as if it does not apply any filter before downscaling. How might this come? Apr 13 '20 at 15:13
• Sure, please just post follow up questions as new questions as they want to avoid such back and forth in the comments. Regarding your referenced implementation, you'll need to pull out and simply what exactly they are doing but you may answer your own question in the process, but in short if you know there is no high frequency content there is no reason to filter.--- Apr 13 '20 at 15:17