0
$\begingroup$

I have a program which does forward and inverse DCT and quantization and dequantization using JPEG matrix. If I apply it to a single image it works fine, but if I try to apply it to a difference between current and previous frame it gets extremely noisy. Album with examples ("high frequency components" line in descriptions means any component starting from anti-diagonal and beyond it).

What is the correct way to compress the difference between two images?

uncompressed

uncompressed

Compressed, high frequency components removed

Compressed, high frequency components removed

Difference between this frame and previous one was coded. High frequency components kept

Difference between this frame and previous one was coded. High frequency components kept

Difference between this frame and previous one was coded. High frequency components removed

Difference between this frame and previous one was coded. High frequency components removed

$\endgroup$
0
$\begingroup$

JPEG was developed for photographic content, not for difference-of-pictures content. Therefore, the fact that it's not overly applicable to the difference between successive frames isn't surprising.

In your specific quantization approach, where you eradicate high frequencies, that's actually even pretty intuitive: The difference between successive frames are probably small, and fine-grained. Of course, when you erase high frequency content, you lose most of the difference information and get a large error. So, I'd call this an indication that your high-frequency deletion works, and little more.

So, wrong approach to video coding; you could use JPEG for I-frames and code the differences to previous frames with some sort of linear predictor. Or, look at existing video codecs, and adapt some of the methods used there - in the case of grass, which presumably is blowing in the wind in a pretty uniform direction with smaller local variations, motion prediction coding might be of special interest to you.

$\endgroup$
3
  • $\begingroup$ "motion prediction coding might be of special interest to you." - that's what I'm working on right now, but motion prediction is conceptually "find the best match in a reference frame, subtract it from the current block", or am I wrong? So, in the end of the day I would still be dealing with difference. $\endgroup$ – Lenassa Jun 11 '20 at 12:02
  • $\begingroup$ motion prediction is "find the best estimate for how you need to move (parts of) your previous frame, then take the difference between shifted previous and current frame, and use a compression method for that difference that suits the nature of the difference data". You're trying to compress a difference, but using an inappropriate method. $\endgroup$ – Marcus Müller Jun 11 '20 at 12:07
  • $\begingroup$ From what I read about existing video codecs, they use DCT for residual blocks too. I guess I'll have to look into details then. Thank you for the help. $\endgroup$ – Lenassa Jun 11 '20 at 12:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.