Method and an apparatus for editing multiple video shots

Patent metadata

Field	Value
Publication number	US-12561862-B2
Application number	US-202218274923-A
Country	US
Kind code	B2
Filing date	Jan 19, 2022
Priority date	Jan 28, 2021
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

A method and an apparatus for editing at least two sequences of pictures are provided. Editing the sequences comprises modifying at least one picture of one of the at least two sequences, based on a user input, and updating at least one picture of at least one other sequence of the at least two sequences, with the modification.

First claim

The invention claimed is:

1 . A video editing method comprising:
determining, for each of at least two video shots, a mosaic representative of the respective video shot, wherein at least one same object appears in pictures of the at least two video shots; determining, for each of the at least two video shots, a transformation between the mosaic representative of the respective video shot and a common embedding space representative of the at least two video shots, wherein determining the transformation between the mosaic representative of the respective video shot and the common embedding space comprises determining color transform and warping from the mosaic to the common embedding space; and responsive to a modification of a first picture of a first video shot of the at least two video shots, updating a second picture of a second video shot of the at least two video shots with the modification using the common embedding space, wherein updating the second picture with the modification is based on the determined color transform and warping.

2 . The method of claim 1 , wherein the modification of the first picture comprises a modification of the at least one same object.

3 . The method of claim 1 , wherein updating the second picture with the modification based on the determined color transform and warping comprises:
projecting the modification from the mosaic representative of the first video shot to the common embedding space using the determined color transform and warping to determine a projected modification; and projecting the projected modification from the common embedding space to a mosaic representative of the second video shot using the determined color transform and warping.

4 . The method of claim 1 , wherein the common embedding space enforces a spatial invariance constraint between each mosaic representative of the at least two video shots.

5 . The method of claim 1 , further comprising determining a visibility map indicating for the mosaic representative of the respective video shot, which part of the mosaic representative of the respective video shot is visible in the common embedding space.

6 . The method of claim 1 , wherein determining the mosaic representative of the respective video shot comprises:
determining 2D trajectories of determined points in the respective video shot; determining an embedding from the 2D trajectories of determined points to a 2D space; and determining the mosaic representative of the respective video shot from the embedding, using a stitching technique.

7 . The method of claim 6 , wherein determining the 2D trajectories of determined points in the respective video shot is based on a 3D model of the same object projected onto each picture of the respective video shot.

8 . The method of claim 1 , further comprising:
determining a spatial relationship between each picture of the respective video shot and the mosaic representative of the respective video shot; determining a color correction model for each picture of the respective video shot from the mosaic representative of the respective video shot; and determining visibility information indicating, for each picture, which part of the mosaic representative of the respective video shot is visible in the picture.

9 . The method of claim 1 , wherein the common embedding space is determined using a neural network.

10 . The method of claim 1 , wherein the common embedding space is determined using local linear embedding.

11 . An apparatus comprising:
a processor configured to: determine, for each of at least two video shots, a mosaic representative of the respective video shot, wherein at least one same object appears in pictures of the at least two video shots; determine, for each one of the at least two video shots, a transformation between the mosaic representative of the respective video shot and a common embedding space representative of the at least two video shots, wherein the processor configured to determine the transformation between the mosaic representative of the respective video shot and the common embedding space comprises the processor being configured to determine color transform and warping from the mosaic to the common embedding space; and responsive to a modification of a first picture of a first video shot of the at least two video shots, update a second picture of a second video shot of the at least two video shots with the modification using the common embedding space, wherein the processor configured to update the second picture with the modification is based on the determined color transform and warping.

12 . The apparatus of claim 11 , wherein the modification of the first picture comprises a modification of the at least one same object.

13 . The apparatus of claim 11 , wherein the processor configured to update the second picture with the modification based on the determined color transform and warping comprises the processor being configured to:
project the modification from the mosaic representative of the first video shot to the common embedding space using the determined color transform and warping to determine a projected modification; and project the projected modification from the common embedding space to a mosaic representative of the second video shot using the determined color transform and warping.

14 . The apparatus of claim 11 , wherein the common embedding space enforces a spatial invariance constraint between each mosaic representative of the at least two video shots.

15 . The apparatus of claim 11 , wherein the processor is further configured to determine a visibility map indicating for the mosaic representative of the respective video shot, which part of the mosaic representative of the respective video shot is visible in the common embedding space.

16 . The apparatus of claim 11 , wherein the processor configured to determine the mosaic representative of the respective video shot comprises the processor being configured to:
determine 2D trajectories of determined points in the respective video shot; determine an embedding from the 2D trajectories of determined points to a 2D space; and determine the mosaic representative of the respective video shot from the embedding, using a stitching technique.

17 . The apparatus of claim 11 , wherein the processor is further configured to:
determine a spatial relationship between each picture of the respective video shot and the mosaic representative of the respective video shot; determine a color correction model for each picture of the video shot from the mosaic representative of the respective video shot; and determine visibility information indicating, for each picture, which part of the mosaic representative of the respective video shot is visible in the picture.

18 . A non-transitory computer readable storage medium having stored thereon instructions for causing one or more processors to perform the method of claim 1 .

Assignees

• Interdigital Madison Patent Holdings Sas

Inventors

• Hellier Pierre
• Jolly Emmanuel
• Damodaran Bharath Bhushan

Classifications

• H04N1/60

  Colour correction or control {(H04N1/54 takes precedence)} · CPC title

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T3/18

  Image warping, e.g. rearranging pixels individually · CPC title

• G06V10/56

  relating to colour · CPC title