Preparation systems for efficiently generating alpha mattes and modified digital videos utilizing polarized light

Patent metadata

Field	Value
Publication number	US-12387410-B2
Application number	US-202318177491-A
Country	US
Kind code	B2
Filing date	Mar 2, 2023
Priority date	Mar 2, 2023
Publication date	Aug 12, 2025
Grant date	Aug 12, 2025

Abstract

The present disclosure relates to systems, non-transitory computer-readable media, and methods for efficiently automating the preparation of accurate alpha matte animations and modified digital videos utilizing polarized light. For example, the disclosed systems obtain a plurality of polarized digital videos portraying an animation of a foreground subject backlit by a polarized light source. In some embodiments, the disclosed systems generate a plurality of corrected polarized digital videos by adjusting intensity values of the plurality of polarized digital videos based on intensity differences across the plurality of polarized digital videos. The disclosed systems generate an alpha matte animation comprising a plurality of alpha mattes from the plurality of corrected polarized digital videos or from the plurality of polarized digital videos. Utilizing the alpha matte animation, the disclosed systems generate a modified digital video by combining the animation of the foreground subject and a replacement background.

First claim

What is claimed is:

1. A method comprising:
obtaining a plurality of polarized digital videos portraying an animation of a foreground subject backlit by a polarized light source;
generating a plurality of corrected polarized digital videos by adjusting intensity values of the plurality of polarized digital videos based on intensity differences across the plurality of polarized digital videos;
generating an alpha matte animation comprising a plurality of alpha mattes from the plurality of corrected polarized digital videos; and
generating a modified digital video by combining the animation of the foreground subject and a replacement background utilizing the alpha matte animation.

2. The method of claim 1 , wherein obtaining the plurality of polarized digital videos comprises:
identifying a digital video portraying the animation of the foreground subject backlit by the polarized light source; and
extracting the plurality of polarized digital videos from the digital video.

3. The method of claim 1 , wherein the plurality of polarized digital videos comprises marker frames and non-marker frames, wherein the marker frames portray the polarized light source having a plurality of markers, and wherein the method further comprises spatially cropping the plurality of polarized digital videos utilizing the plurality of markers from the polarized light source portrayed in the marker frames.

4. The method of claim 3 , further comprising temporally cropping the plurality of polarized digital videos by removing the marker frames from the plurality of polarized digital videos.

5. The method of claim 1 , wherein obtaining the plurality of polarized digital videos comprises:
obtaining a first polarized digital video comprising polarized light at a first angle;
obtaining a second polarized digital video comprising polarized light at a second angle orthogonal to the first angle;
obtaining a third polarized digital video comprising polarized light at a third angle; and
obtaining a fourth polarized digital video comprising polarized light at a fourth angle orthogonal to the third angle.

6. The method of claim 5 , wherein generating the plurality of corrected polarized digital videos comprises, for a first pixel of a first frame of the plurality of polarized digital videos:
generating a first intensity correction metric based on intensity values for corresponding first pixels of corresponding first frames of the first polarized digital video, the second polarized digital video, the third polarized digital video, and the fourth polarized digital video; and
generating, based on the first intensity correction metric, adjusted intensity values for the corresponding first pixels of the corresponding first frames of the first polarized digital video, the second polarized digital video, the third polarized digital video, and the fourth polarized digital video.

7. The method of claim 6 , wherein generating the plurality of corrected polarized digital videos comprises, for a second pixel of the first frame of the plurality of polarized digital videos:
generating a second intensity correction metric based on intensity values for corresponding second pixels of the corresponding first frames of the first polarized digital video, the second polarized digital video, the third polarized digital video, and the fourth polarized digital video; and
generating, based on the second intensity correction metric, adjusted intensity values for the corresponding second pixels of the corresponding first frames of the first polarized digital video, the second polarized digital video, the third polarized digital video, and the fourth polarized digital video.

8. The method of claim 5 , wherein adjusting the intensity values of the plurality of polarized digital videos comprises solving a closed-form expression of intensity values for the first polarized digital video, the second polarized digital video, the third polarized digital video, and the fourth polarized digital video.

9. The method of claim 1 , further comprising generating a training dataset, wherein generating the modified digital video comprises generating a machine-learning training video for the training dataset, and wherein generating the alpha matte animation comprises generating a ground truth alpha matte animation for the machine-learning training video.

10. A system comprising:
a memory component; and
one or more processing devices coupled to the memory component, the one or more processing devices to perform operations comprising:
obtaining a plurality of polarized digital videos portraying an animation of a foreground subject and a polarized light source;
generating a plurality of correction metrics by comparing pixel-wise intensity values across corresponding frames of the plurality of polarized digital videos;
generating a plurality of corrected polarized digital videos by adjusting intensity values of the plurality of polarized digital videos utilizing the plurality of correction metrics;
generating a plurality of alpha mattes from the plurality of corrected polarized digital videos; and
generating a modified digital video from the animation of the foreground subject and the plurality of alpha mattes.

11. The system of claim 10 , wherein obtaining the plurality of polarized digital videos comprises capturing, utilizing one or more polarized filters of a camera, a digital video portraying the animation of the foreground subject and the polarized light source.

12. The system of claim 10 , wherein obtaining the plurality of polarized digital videos comprises:
obtaining a first polarized digital video comprising polarized light at a first angle;
obtaining a second polarized digital video comprising polarized light at a second angle orthogonal to the first angle and orthogonal to a polarization angle of the polarized light source;
obtaining a third polarized digital video comprising polarized light at a third angle; and
obtaining a fourth polarized digital video comprising polarized light at a fourth angle orthogonal to the third angle.

13. The system of claim 12 , wherein generating the modified digital video from the animation of the foreground subject and the plurality of alpha mattes comprises combining the animation of the foreground subject as displayed in the second polarized digital video with a replacement background utilizing the plurality of alpha mattes.

14. The system of claim 10 , wherein generating the plurality of correction metrics comprises generating a first intensity correction metric based on:
intensity values for corresponding first pixels of corresponding first frames of the plurality of polarized digital videos; and
a first overall intensity value for the corresponding first pixels of the corresponding first frames of the plurality of polarized digital videos; and

wherein generating the plurality of corrected polarized digital videos comprises generating, based on the first intensity correction metric, a first set of adjusted intensity values for the corresponding first pixels of the corresponding first frames of the plurality of polarized digital videos.

15. The system of claim 14 , wherein generating the plurality of correction metrics comprises generating a second intensity correction metric based on:
intensity values for corresponding second pixels of corresponding second frames of the plurality of polarized digital videos; and
a second overall intensity value for the corresponding second pixels of the corresponding second frames of the plurality of polarized digital videos; and

wherein generating the plurality of correc

Assignees

• Adobe Inc

Inventors

• Rhodes Tenell
• Price Brian
• Enomoto Kenji
• Wampler Kevin

Classifications

• G06T7/194

  involving foreground-background segmentation · CPC title

• G06V10/60 Primary

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06V10/771

  Feature selection, e.g. selecting representative features from a multi-dimensional feature space · CPC title

• H04N5/272

  Means for inserting a foreground image in a background image, i.e. inlay, outlay · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title