I have a program which does forward and inverse DCT and quantization and dequantization using JPEG matrix. If I apply it to a single image it works fine, but if I try to apply it to a difference between current and previous frame it gets extremely noisy. Album with examples ("high frequency components" line in descriptions means any component starting from anti-diagonal and beyond it).
What is the correct way to compress the difference between two images?
Compressed, high frequency components removed
Difference between this frame and previous one was coded. High frequency components kept
Difference between this frame and previous one was coded. High frequency components removed