Systems and methods for propagating edits through a video

Patent metadata

Field	Value
Publication number	US-9911215-B1
Application number	US-201615343677-A
Country	US
Kind code	B1
Filing date	Nov 4, 2016
Priority date	Nov 4, 2016
Publication date	Mar 6, 2018
Grant date	Mar 6, 2018

Abstract

Systems and methods of propagating edits through a video are presented herein. Decompositions of information defining frame images of a video may be obtained. Individual decompositions may include individual sets of sinusoidal functions separated into individual frequency bands. One or more frequency bands that convey differences between an (unedited) frame image and an edited version of that frame image may be identified. A subset of the one or more frequency bands that convey motion information associated with the frame image and a subsequent frame image in the video may be identified. Using the motion information, phase information of individual sinusoidal functions of a decomposition of the edited frame image, which may be included in the subset of the one or more frequency bands, may be propagated based on the motion information.

First claim

What is claimed is:

1. A system configured to propagate edits through a video, the system comprising:
one or more physical processors configured by machine-readable instructions to:
obtain decompositions of information defining frame images of a video,

individual decompositions including individual sets of sinusoidal functions, individual sinusoidal functions in the individual sets of sinusoidal functions being separated into individual frequency bands, individual sinusoidal functions within an individual frequency band having amplitude information and phase information, the decompositions including a first decomposition of a first frame image, a second decomposition of an edited version of the first frame image, and a third decomposition of a second frame image;
identify, from the first decomposition and the second decomposition, one or more frequency bands that convey differences between the first frame image and the edited first frame image;
identify, from the one or more frequency bands that convey differences between the first frame image and the edited first frame image, a subset of the one or more frequency bands that convey motion information associated with the first frame image and the second frame image;
determine, from sinusoidal functions of the first decomposition and third decomposition that are included in the subset of the one or more frequency bands, the motion information; and
propagate phase information of individual sinusoidal functions of the second decomposition that are included in the subset of the one or more frequency bands based on the motion information.

2. The system of claim 1 , wherein identifying the one or more frequency bands that convey differences between the first frame image and the edited first frame image based on the first decomposition and the second decomposition comprises:
determining that differences in amplitude information between the one or more frequency bands of the first decomposition and the second decomposition are greater than a threshold.

3. The system of claim 2 , wherein the one or more differences are normalized differences.

4. The system of claim 1 , wherein identifying the subset of the one or more frequency bands comprises:
determining that amplitude information in the subset of the one or more frequency bands in the first decomposition and the third decomposition is greater than a threshold.

5. The system of claim 1 , wherein the motion information is determined based on differences between phase information included in the subset of the one or more frequency bands of the first decomposition and third decomposition.

6. The system of claim 1 , wherein propagating the phase information of individual sinusoidal functions of the second decomposition that are included in the subset of the one or more frequency bands based on the motion information defines a fourth decomposition of information defining an edited version of the second frame image.

7. The system of claim 6 , wherein the one or more physical processors are further configured by machine-readable instructions to:
identify a second subset of the one or more frequency bands that convey differences between the first frame image and the edited first frame image not included in the subset of the one or more frequency bands, the second subset comprising one or more frequency bands that do not convey motion information associated with the first image and the second image;
for the second subset of the one or more frequency bands, determine estimated motion information; and
propagate phase information of individual sinusoidal functions of the second decomposition that are included in the second subset of the one or more frequency bands to individual sinusoidal functions of the fourth decomposition that are also included in the second subset of the one or more frequency bands, the propagation being based on the estimated motion information.

8. The system of claim 7 , wherein the estimated motion information is determined from motion information derived from one or more other frequency bands not included in the second subset.

9. The system of claim 7 , wherein the one or more physical processors are further configured by machine-readable instructions to:
determine the information defining the edited version of the second frame image from the fourth decomposition of the information defining the edited version of the second frame image;
determine a fifth decomposition of the information defining the edited version of the second frame image, the fifth decomposition being different from the fourth decomposition;
compare the fifth decomposition with the third decomposition to determine differences in amplitude information included in the second subset of the one or more frequency bands,
for one or more frequency bands in the second subset of frequency bands that include amplitude information for the fifth decomposition that is larger than amplitude information for the third decomposition in corresponding ones of the one or more frequency bands, increase the amplitudes of one or more sinusoidal functions in the corresponding ones of the one or more frequency bands in the second subset of frequency bands for the fifth decomposition.

10. The system of claim 1 , wherein the decompositions are obtained by applying a pyramid filter.

11. A method of propagating edits through a video, the method being implemented in a computer system comprising one or more physical processors and non-transitory electronic storage storing machine-readable instructions, the method comprising:
obtaining decompositions of information defining frame images of a video, individual decompositions including individual sets of sinusoidal functions, individual sinusoidal functions in the individual sets of sinusoidal functions being separated into individual frequency bands, individual sinusoidal functions within an individual frequency band having amplitude information and phase information, including obtaining a first decomposition of a first frame image, a second decomposition of an edited version of the first frame image, and a third decomposition of a second frame image;
identifying, from the first decomposition and the second decomposition, one or more frequency bands that convey differences between the first frame image and the edited first frame image;
identifying, from the one or more frequency bands that convey differences between the first frame image and the edited first frame image, a subset of the one or more frequency bands that convey motion information associated with the first frame image and the second frame image;
determining, from sinusoidal functions of the first decomposition and third decomposition that are included in the subset of the one or more frequency bands, the motion information; and
propagating phase information of individual sinusoidal functions of the second decomposition that are included in the subset of the one or more frequency bands based on the motion information.

12. The method of claim 11 , wherein identifying the one or more frequency bands that convey differences between the first frame image and the edited first frame image based on the first decomposition and the second decomposition comprises:
determining that differences in amplitude information between the one or more frequency bands of the first decomposition and the second decomposition are greater than a threshold.

13. The method of claim 12 , wherein the one or more differences are normalized differences.

14. The method of claim 11 , wherein identifying the subset of the one or more frequency bands comprises:
determining that amplitude informat

Assignees

• Disney Entpr Inc
• Eth Zuerich

Inventors

• Hornung Alexander Sorkine
• Meyer Simone

Classifications

• G11B27/031 Primary

  Electronic editing of digitised analogue information signals, e.g. audio or video signals · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T7/0022

  Physics · mapped topic

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title