4
$\begingroup$

I'm looking to composite an opaque object (e.g. 3D art) into video footage of semitransparent fog/clouds before a landscape background so that the object seems immersed in the clouds but occludes the background. Ideally this would mean separating the footage with the clouds and landscape into two layers.

What is the state of the art in matting/masking/extracting transparent objects from video frames? Any depth mask/inter-frame/AI algorithms available? I'm a software developer, but any suggestions for existing (experimental) tools are also welcome.

Improve this question
$\endgroup$
4
  • $\begingroup$ Could you edit your question for more clarity? It sounds to me like you have one set of footage with clouds against a background, and another with your solid object. So you essentially want to separate that first set of footage into two layers (background and clouds), and then insert your solid object in between. Is that correct? $\endgroup$
    – TimWescott
    Commented Jun 23, 2023 at 16:41
  • $\begingroup$ While you're editing -- do you want to be pointed to the technology so that you can write an application that does this and then you do post-editing, or are you looking for a video editing tool that has this technology built in? $\endgroup$
    – TimWescott
    Commented Jun 23, 2023 at 16:42
  • $\begingroup$ @TimWescott thanks for the suggestion, I've updated the question. $\endgroup$
    – LogicBreaker
    Commented Jun 23, 2023 at 17:17
  • $\begingroup$ @LogicBreaker, Could you please review my answer? $\endgroup$
    – Royi
    Commented Jul 3, 2023 at 13:04

1 Answer 1

Reset to default
1
$\begingroup$

One of the current state of the art methods is based on the U-Net Square model.

Have a look on their examples:

enter image description here

enter image description here

They can handle semi transparent cases.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.