Method, apparatus and stream for immersive video format

Patent metadata

Field	Value
Publication number	US-10891784-B2
Application number	US-201816477620-A
Country	US
Kind code	B2
Filing date	Jan 8, 2018
Priority date	Jan 13, 2017
Publication date	Jan 12, 2021
Grant date	Jan 12, 2021

Abstract

Method and device for generating a stream of data representative of a 3D point cloud. The 3D point cloud is partitioned into a plurality of 3D elementary parts. A set of two-dimensional 2D parametrizations is determined, each 2D parametrization representing one 3D part of the point cloud with a set of parameters. Each 3D part is represented as a 2D pixel image. A depth map and a color map are determined as a first patch atlas and a second patch atlas. A data stream is generated by combining and/or coding the parameters of the 2D parametrization, the first patch atlas, the second patch atlas and mapping information that links each 2D parametrization with its associated depth map and color map in respectively the first and second patch atlas.

First claim

The invention claimed is:

1. A method of generating a stream, the method comprising:
partitioning a three-dimensional point cloud in a plurality of three-dimensional parts each comprising at least a point of the three-dimensional point cloud;
for each of said three-dimensional parts:
determining a depth map of said three-dimensional part according to parameters representative of a two-dimensional parametrization responsive to geometric information associated with said at least a point and to pose information associated with a range of points of view; and

determining a color map of said three-dimensional part according to said parameters;
the plurality of determined depth maps being represented in a first patch atlas, each depth map corresponding to one patch of the first patch atlas, and the plurality of determined color maps being represented in a second patch atlas, each color map corresponding to one patch of the second patch atlas;
generating said stream comprising the parameters representative of the two-dimensional parametrizations, data representative of the first patch atlas, data representative of the second patch atlas and a first mapping information representative of a mapping between said two-dimensional parametrizations and corresponding depth maps in the first patch atlas and corresponding color maps in the second patch atlas.

2. A device configured to generate a stream, the device comprising at least one processor and at least one memory having stored thereon instructions operative, when executed by the at least one processor to cause the device to:
partition a three-dimensional point cloud in a plurality of three-dimensional parts each comprising at least a point of the three-dimensional point cloud;
determine, for each of said three-dimensional parts:
a depth map of said three-dimensional part according to parameters representative of a two-dimensional parametrization responsive to geometric information associated with said at least a point and to pose information associated with a range of points of view; and
a color map of said three-dimensional part according to said parameters;

the plurality of determined depth maps being represented in a first patch atlas, each depth map corresponding to one patch of the first patch atlas, and the plurality of determined color maps being represented in a second patch atlas, each color map corresponding to one patch of the second patch atlas;
generate said stream comprising the parameters representative of the two-dimensional parametrizations, data representative of the first patch atlas, data representative of the second patch atlas and a first mapping information representative of a mapping between said two-dimensional parametrizations and corresponding depth maps in the first patch atlas and corresponding color maps in the second patch atlas.

3. The method according to claim 1 , wherein a plurality of two-dimensional parametrizations is associated with each three-dimensional part of at least a part of said plurality of three-dimensional parts.

4. The method according to claim 1 , wherein said stream further comprises a second mapping information representative of a mapping between an information identifying said three-dimensional patches and pixels of first and second images representative of respectively the first patch atlas and the second patch atlas.

5. The method according to claim 1 , wherein the partitioning of said point cloud varies over time.

6. A method of obtaining a point cloud from a stream carrying data representative of said object, the method comprising:
obtaining, from said stream, parameters representative of two-dimensional parametrizations of three-dimensional parts of a point cloud;
obtaining, from said stream, data representative of a first patch atlas comprising a plurality of depth maps each associated with one of said two-dimensional parametrizations;
obtaining, from said stream, data representative of a second patch atlas comprising a plurality of color maps each associated with one of said two-dimensional parametrizations;
obtaining data associated with points of the point cloud from said parameters representative of the two-dimensional parametrizations, data of associated depth maps in the first patch atlas, data of associated color maps in the second patch atlas and first mapping information obtained from said stream and representative of a mapping between said two-dimensional parametrizations and corresponding depth maps and color maps in respectively said first patch atlas and said second patch atlas.

7. A device configured to obtain a point cloud from a stream carrying data representative of said object, the device comprising at least one processor and at least one memory having stored thereon instructions operative, when executed by the at least one processor, to cause the device to:
obtain, from said stream, parameters representative of two-dimensional parametrizations of three-dimensional parts of a point cloud;
obtain, from said stream, data representative of a first patch atlas comprising a plurality of depth maps each associated with one of said two-dimensional parametrizations;
obtain, from said stream, data representative of a second patch atlas comprising a plurality of color maps each associated with one of said two-dimensional parametrizations;
obtain data associated with points of the point cloud from said parameters representative of the two-dimensional parametrizations, data of associated depth maps in the first patch atlas, data of associated color maps in the second patch atlas and first mapping information obtained from said stream and representative of a mapping between said two-dimensional parametrizations and corresponding depth maps and color maps in respectively said first patch atlas and said second patch atlas.

8. The method according to claim 6 , wherein a plurality of two-dimensional parametrizations is associated with each three-dimensional part of at least a part of the plurality of three-dimensional parts.

9. The method according to claim 6 , wherein said stream further comprises a second mapping information representative of a mapping between the three-dimensional parts and pixels of first and second images representative of respectively the first patch atlas and the second patch atlas.

10. The method according to claim 6 , wherein an image representative of the object is rendered according to the obtained data representative of the point cloud.

11. A non-transitory processor-readable storage medium having stored therein instructions that, when executed by a processor, cause the processor to:
partition a three-dimensional point cloud in a plurality of three-dimensional parts each comprising at least a point of the three-dimensional point cloud;
for each of said three-dimensional parts:
determine a depth map of said three-dimensional part according to parameters representative of a two-dimensional parametrization responsive to geometric information associated with said at least a point and to pose information associated with a range of points of view; and
determine a color map of said three-dimensional part according to said parameters;

the plurality of determined depth maps being represented in a first patch atlas, each depth map corresponding to one patch of the first patch atlas, and the plurality of determined color maps being represented in a second patch atlas, each color map corresponding to one patch of the second batch atlas;
generate said stream comprising the parameters representative of the two-dimensional parametrizations, data representative of the first patch atlas, data representative of the second patch atlas and a first mapping information

Assignees

• Interdigital Vc Holdings Inc

Inventors

• Dore Renaud
• GALPIN Franck
• Briand Gerard
• Fleureau Julien

Classifications

• H04N19/597

  specially adapted for multi-view video sequence encoding · CPC title

• H04N19/20

  using video object coding · CPC title

• H04N13/279

  the virtual viewpoint locations being selected by the viewers or determined by tracking · CPC title

• G06T2200/08

  involving all processing steps from image acquisition to 3D model generation · CPC title

• G06T2200/04

  involving 3D image data · CPC title