Camerawork generating method and video processing device

Patent metadata

Field	Value
Publication number	US-2019174109-A1
Application number	US-201916270931-A
Country	US
Kind code	A1
Filing date	Feb 8, 2019
Priority date	Aug 10, 2016
Publication date	Jun 6, 2019
Grant date	—

Abstract

A camerawork generating method includes: generating, from a multi-view video of a same scene captured from different viewpoints, a three-dimensional model for the scene; determining a target scene which is included in the scene and is a target for which a free-viewpoint video of the three-dimensional model as viewed from a virtual camera is to be generated; and generating a camerawork representing a temporal change in a position and an orientation of the virtual camera in the free-viewpoint video.

First claim

What is claimed is:

1 . A camerawork generating method, comprising:
generating, from a multi-view video of a same scene captured from different viewpoints, a three-dimensional model for the scene; determining a target scene that is a target for which a free-viewpoint video of the three-dimensional model as viewed from a virtual camera is to be generated, the target scene being included in the scene; and generating a camerawork representing a temporal change in a position and an orientation of the virtual camera in the free-viewpoint video.

2 . The camerawork generating method according to claim 1 , wherein
in the determining of the target scene, the target scene is determined using one of the multi-view video and the three-dimensional model.

3 . The camerawork generating method according to claim 1 , wherein
in the determining of the target scene, the target scene is determined using sound of the scene.

4 . The camerawork generating method according to claim 1 , wherein
in the generating of the camerawork, the position and the orientation of the virtual camera are determined such that an object of a predetermined type associated with the target scene is included in the free-viewpoint video.

5 . The camerawork generating method according to claim 1 , wherein
in the generating of the camerawork, the position and the orientation of the virtual camera are determined based on precisions of a plurality of portions included in the three-dimensional model.

6 . The camerawork generating method according to claim 1 , further comprising:
displaying the camerawork.

7 . The camerawork generating method according to claim 1 , further comprising:
generating the free-viewpoint video from one of the three-dimensional model and the multi-view video, using the camerawork.

8 . The camerawork generating method according to claim 7 , wherein
in the generating of the free-viewpoint video, a shake is added to the free-viewpoint video.

9 . The camerawork generating method according to claim 1 , further comprising:
acquiring an evaluation of the camerawork by a user, wherein in the generating of the camerawork, the camerawork is generated based on the evaluation.

10 . A video processing device, comprising:
a model generation unit that generates, from a multi-view video of a same scene captured from different viewpoints, a three-dimensional model for the scene; an analyzer that determines a target scene that is a target for which a free-viewpoint video of the three-dimensional model as viewed from a virtual camera is to be generated, the target scene being included in the scene; and a camerawork generation unit that generates a camerawork representing a temporal change in a position and an orientation of the virtual camera in the free-viewpoint video.

Assignees

• Panasonic Ip Corp America

Inventors

• Yoshikawa Satoshi
• Sugio Toshiyasu
• Matsunobu Toru
• Koyama Tatsuya
• SASAKURA Shuuhei
• Yamashita Chisato

Classifications

• H04N13/243

  using three or more two-dimensional [2D] image sensors · CPC title

• G06T15/20

  Perspective computation · CPC title

• H04N21/234

  Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs · CPC title

• H04N21/218

  Source of audio or video content {, e.g. local disk arrays (details of retrieval in video databases G06F16/739)} · CPC title

• H04N13/279

  the virtual viewpoint locations being selected by the viewers or determined by tracking · CPC title