Upscaling detection approach

I'm trying to implement a solution that would, to some extent, evaluate a picture telling what are the chances that it was upscaled from the smaller image.

My approach is as follows:

1. Take a picture of unknown origin (scaled or not)
2. Perform sharpening using some technique which sharpens only evident edges (1)
3. Extract edges using Difference of Gaussians
4. Measure entropy of resulting image

Then downscale the image and upscale it back to the same size and compare entropies. OR Then upscale the image and downscale it back to the same size and compare entropies.

Does it sound reasonable? What technique should I use as (1)? Also, maybe I should do it in small steps so I don't introduce too many artificial points? I mean if the image was upscaled (preferably instantly, to target size) it won't matter a lot - but it the image weren't upscaled this incremental improvement will be better for the picture than taking it and upscaling instantly?

• A stretched signal will probably have attenuated high frequencies, have a look at the spectrum? Mar 1 '15 at 12:33
• Yeah and this is what my current approach is based on. I take a picture (either scaled or not) and perform 2D FFT on it. I downscale this image getting a new one and I perform same operation on it. Then I downsample the first amplitude spectrum to get the same number of samples the second has. Then I compute a score for each of such spectrums - it's a sum of products of the radius and value of each point in the diagram. Given that I compute relative factor (not_scaled_ratio - scaled_ratio / not_scaled_ratio) to get a final score. It appears to be negative for nonscaled and positive for scaled. Mar 1 '15 at 13:44

Here is the poor man's approach:

var histogram = []
foreach pixel
histogram.append(abs(pixel.value - pixel.south.value))
histogram.append(abs(pixel.value - pixel.east.value))
sort(histogram)
sharpness = histogram[histogram.count * 0.95]
upscaled = sharpness < X // You choose X experimentally

Given that upscaling methods can be post-processed, and that the data can be rounded or quantized, perfect detection sounds difficult. However, if basic upscaling methods have been used, with polynomial interpolation (linear, quadratic, cubic), then the image could have almost piecewise polynomial sections. Piecewise polynomial parts are quite unnatural in real worlds images.

Then, different biorthogonal wavelets with some vanishing moments (low degree polynomials are invisible to wavelets) could be used. Wavelet subbands could exhibit, at least visually, close-to-zero groups of coefficients, blocks of variance different than the natural noise, or vertical and horizontal stripes like Mondrian painting. This is illustrated below. This is not really an image, but a geological mesh cast as an image. The yellow stripes told us that the so-called fine mesh was indeed upscaled by a factor of $5$ with a simple linear interpolation, which we could verify on the original data. This might be less evident with bi-cubics and quantization, however.

This is an instance of image forgery detection. Image Forgery Detection, A survey by Hany Farid (chapter Resampling) could give you other ideas.

I am not really sure how it works but I used this and works okay. Perhaps you can try it and figure out how it's done?

http://rest7.com/image_upscaled

There you can upload your image and will get original dimensions, like this:

{
"is_upscaled": true,
"current_width": "2000",
"current_height": "928",
"original_width": "1750",
"original_height": "696",
"accuracy": "82%",
"success": 1
}

A simple method for comparing images is template matching

Its usually used for finding a smaller image in a larger image.

You could also use it as a way of comparing how similar two images of the same size are.

The opencv article I linked to lists a number of different correlation functions such as sum of squared differences and correlation coefficient.

Whether or not this will be accurate enough for what you are doing will only become clear after you have tried it.

• But how do I detect if the image was upscaled? I don't have 2 images - just one - and I need to tell if it was stretched. Mar 1 '15 at 8:28
• ah, ok, do you know the scaling algorithm?
– sav
Mar 1 '15 at 22:56
• The problem is - I don't. Should I analyze each upscaling method and try to apply "the reverse"? If it somehow matches (how do I know) than the image has been resized? How about sharpening, which may occur after somebody resized the picture? And plenty of other possible filters... maybe I ask to much? Maybe I need to focus on some particular scenario? Of course it would be best if it worked for all, but if it just can't... Mar 2 '15 at 7:06
• So you are just looking for streching artifact ? because if an image is perfectly stretched, there is no way to say if has been stretched. Apr 30 '15 at 15:10