Upscaling detection approach

Question

I'm trying to implement a solution that would, to some extent, evaluate a picture telling what are the chances that it was upscaled from the smaller image.

My approach is as follows:

1. Take a picture of unknown origin (scaled or not)
2. Perform sharpening using some technique which sharpens only evident edges (1)
3. Extract edges using Difference of Gaussians
4. Measure entropy of resulting image

Then downscale the image and upscale it back to the same size and compare entropies. OR Then upscale the image and downscale it back to the same size and compare entropies.

Does it sound reasonable? What technique should I use as (1)? Also, maybe I should do it in small steps so I don't introduce too many artificial points? I mean if the image was upscaled (preferably instantly, to target size) it won't matter a lot - but it the image weren't upscaled this incremental improvement will be better for the picture than taking it and upscaling instantly?

Answer 1

Here is the poor man's approach:

var histogram = []
foreach pixel
    histogram.append(abs(pixel.value - pixel.south.value))
    histogram.append(abs(pixel.value - pixel.east.value))
sort(histogram)
sharpness = histogram[histogram.count * 0.95]
upscaled = sharpness < X // You choose X experimentally

Answer 2

Given that upscaling methods can be post-processed, and that the data can be rounded or quantized, perfect detection sounds difficult. However, if basic upscaling methods have been used, with polynomial interpolation (linear, quadratic, cubic), then the image could have almost piecewise polynomial sections. Piecewise polynomial parts are quite unnatural in real worlds images.

Then, different biorthogonal wavelets with some vanishing moments (low degree polynomials are invisible to wavelets) could be used. Wavelet subbands could exhibit, at least visually, close-to-zero groups of coefficients, blocks of variance different than the natural noise, or vertical and horizontal stripes like Mondrian painting. This is illustrated below.

This is not really an image, but a geological mesh cast as an image. The yellow stripes told us that the so-called fine mesh was indeed upscaled by a factor of $5$ with a simple linear interpolation, which we could verify on the original data. This might be less evident with bi-cubics and quantization, however.

This is an instance of image forgery detection. Image Forgery Detection, A survey by Hany Farid (chapter Resampling) could give you other ideas.

Answer 3

I am not really sure how it works but I used this and works okay. Perhaps you can try it and figure out how it's done?

http://rest7.com/image_upscaled

There you can upload your image and will get original dimensions, like this:

{
  "is_upscaled": true,
  "current_width": "2000",
  "current_height": "928",
  "original_width": "1750",
  "original_height": "696",
  "accuracy": "82%",
  "success": 1
}

Answer 4

A simple method for comparing images is template matching

Its usually used for finding a smaller image in a larger image.

You could also use it as a way of comparing how similar two images of the same size are.

The opencv article I linked to lists a number of different correlation functions such as sum of squared differences and correlation coefficient.

Whether or not this will be accurate enough for what you are doing will only become clear after you have tried it.