2
$\begingroup$

For a project I'm working on (master thesis), I need to figure out what is facebook's image compression algorithm.

The goal: to be able to mimic facebook's image compression locally (using python).

The images I use have the following exif: Image exif (Q = 90) After facebook compression, I have the following exif : Compressed image exif

Note: If the images I uploaded with the link seem unavailable, please refresh the page! :)

After searching for a while, I could not find a recent and clear answer to my question. I tried to run the compressed image through ImageMagick's identify. It tells me that it is a JPEG compression (DCT progressive + Huffman) with a quality of 85. But when I try to do my own compression using python, the result is not the same.

Here is my code:

from PIL import Image
img = Image.open('original.jpg')
img.save('output.jpg', optimize=True, quality=85, progressive=True)

Do you have any ideas or resources that could help me? I'm running out of ideas. Thank you!

Improve this question
$\endgroup$
7
  • $\begingroup$ the EXIF tag has absolutely nothing to do with the compression used. So, I'm afraid, that's a dead end. Different compressors yield different compression results, so that your PIL and facebook's JPEG compressor delivering different results is not surprising. Also, you might want to look into what your identify tool actually does; you might find it's not what you think it does: there's no canonical mapping of compression settings to percentages – it's all a bit of approximate / perceptive settings. It's quite possible Facebook doesn't even use classical JPEG for storage internally, but $\endgroup$
    – Marcus Müller
    Apr 5, 2021 at 11:04
  • $\begingroup$ something more modern, and just re-encodes it to JPEG for your browser, which might lead to the combined artifacts of the Facebook-internal compression and the "delivery" JPEG compression. I think for a master's thesis you'll be expected to dig a fair bit deeper. Start by understanding the different file formats Facebook will deliver to you (JPEG, WebP, PNG?, GIF?), then start digging into these file formats and extract the quantizer tables. Automate a bit, and do an analysis over a representative corpus of test images, to characterize the quantization decisions made. Craft images to verify $\endgroup$
    – Marcus Müller
    Apr 5, 2021 at 11:05
  • $\begingroup$ your hypothesis. Your supervisor certainly didn't hand out a master thesis with the content "run a tool on the files to tell you what you need to do, then write 20 lines of report and hand it in"! $\endgroup$
    – Marcus Müller
    Apr 5, 2021 at 11:08
  • $\begingroup$ @MarcusMüller Tank you for your answer, in fact my master thesis is not about the compression that facebook is using :) It's just a way to present what my work can lead to. My goal is to train AI to create steganographic images that are resistant to compression. I wanted to find facebook's algorithm to train my model against it. $\endgroup$
    – user2679262
    Apr 5, 2021 at 11:25
  • $\begingroup$ well, then you might need to be even less smart, and simply build an automated toolchain that uploads an image, downloads the compressed variant from facebook, and uses this channel as penalty function in an ML-based autoencoder; this is pretty much a variant of all the ML-based communication modems. You might want to read Tim O'Shea and others on autoencoders. $\endgroup$
    – Marcus Müller
    Apr 5, 2021 at 11:29

2 Answers 2

Reset to default
2
$\begingroup$

If you have access to the bitstream, you should be able to figure out which codec they are using.

Assuming that it is good old jpeg, possibly with a tweaked encoder implementation, I would parse the stream using JPEGSnoop to learn as much as possible. It can even try to match «known» encoders: https://www.impulseadventure.com/photo/jpeg-snoop.html

If you can upload uncompressed bmp images to facebook and observe the first-generation compressed jpeg, you might learn even more as you have a reference.

Reverse engineering an encoder black box like this may be tedious unless they went with a plain vanilla implementation.

JPEG is actually a flexible standard, and «quality=85» is a simplified way of expressing how 8x8 DCT coefficients of luma (and chroma) are quantified. Sniffing the stream will allow you to read out those 2x64 quantization scalings, and also (if any) chroma subsampling, and (unlikely) non-bt601 color primary matrixing. An encoder is allowed to set any (small) dct coeff to zero if it likes (in the interest of smaller files), besides the regular quantization. Also, the encoder may use non-standard huffman tables but that should affect the bitstream not pixels.

Improve this answer
$\endgroup$
2
$\begingroup$

As couple of years ago, I asked a former colleague, who is now working at Facebook on Artificial Intelligence, whether they used internally something else than JPEG. And the answer was: "no".

However, here are a couple of additional sources to investigate the issue, including questions on Facebook processing/enhancing images:

If you have pre-processes image, the image would more likely to become over-processed by FB. In below image, in simple words, you can see the brightness of picture has been increased & the details has been changed and the details as well.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.