3
$\begingroup$

With many very similar frames of video that have been compressed with an unavoidably lossy encoding and available now as jpeg stills, are there available good mathematical techniques for recovering detail by considering multiple frames? Certain areas of the image are of great interest and the artifacts there change from frame to frame. Can they be cross-referenced to solve for higher detail?

$\endgroup$
3
$\begingroup$

The method to improve resolution by few frames is called Multi Image Super Resolution (As opposed to Single Image Super Resolution).

For compression artifacts you can look for JPEG Deblocking algorithms.

$\endgroup$
1
$\begingroup$

As video compression implicitly tries to track scene motion and express the residual approximately in a compact (and visually pleasing manner), I think that applying general multi frame super reolution is unlikely to give good results unless your video stream is encoded using only i-frames (eg motion jpeg, where temporal information is not exploited by the codex) or at very high bitrates (where camera flaws are the main bottleneck).

Any subjective improvement to «normal» lossy video I would expect to be connected to some combination of large training sets, specific encoder behaviour and embedded knowledge about human perception.

$\endgroup$
1
$\begingroup$

Beside Video Super-Resolution, as answered by @Royi,Other words for this task (fill in missing regions of a given video sequence with contents that are both spatially and temporally coherent) are video inpainting, video completion, or even video restoration.

There are many flavors: motion-based, object based, using deep learning, etc. A couple of links

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.