Method and apparatus for encoding and decoding three-dimensional scenes in and from a data stream

Patent metadata

Field	Value
Publication number	US-12249107-B2
Application number	US-202318201013-A
Country	US
Kind code	B2
Filing date	May 23, 2023
Priority date	Jan 19, 2018
Publication date	Mar 11, 2025
Grant date	Mar 11, 2025

Abstract

Methods and devices are provided to encode and decode a data stream carrying data representative of a three-dimensional scene, the data stream comprising color pictures packed in a color image; depth pictures packed in a depth image; and a set of patch data items comprising de-projection data; data for retrieving a color picture in the color image and geometry data. Two types of geometry data are possible. The first type of data describes how to retrieve a depth picture in the depth image. The second type of data comprises an identifier of a parametric function and a list of parameter values for the identified parametric function.

First claim

The invention claimed is:

1. A method comprising:
obtaining from a data stream:
a color patch atlas packing color pictures;
a depth patch atlas packing depth pictures corresponding to color patches of a given geometry type; and
metadata comprising, for each color patch of the color patch atlas:
de-projection data;
a color data comprising a description of a location of a color picture in the color image; and
an information indicating whether the color picture is of the given geometry type; and
if the information indicates that the color picture is of the given geometry type, a description of a location of a depth picture in the depth patch atlas; and
if the information indicates that the color picture is not of the given geometry type, a constant depth value; and

for each color patch of the given geometry type:
using de-projection data for de-projecting pixels of the color picture described by the color data at a depth value determined according to pixels of the corresponding depth patch in the depth atlas; and

for each other color patch:
using de-projection data for de-projecting pixels of the color picture described by the color data at the constant depth value in the associated metadata.

2. The method of claim 1 , wherein a color data further comprises a description of at least one of a size or of a shape of at least one color patch in the color patch atlas.

3. The method of claim 1 , wherein the data stream comprises a sequence of color patch atlases, a sequence of depth patch atlases and associated metadata, each depth patch atlas and related metadata being associated with a color patch atlas, wherein the color patch atlases are structured by temporal information.

4. A method comprising:
obtaining a set of color patches, and, for each color patch, an information indicating whether the color patch is of a given geometry type, and if so, an associated depth patch, and if not, an associated constant depth value;
generating a color patch atlas by packing the color patches in a color patch atlas and generating a color data comprising a description of a location of each color patch as packed in the color patch atlas;
generating a depth patch atlas by packing depth patches associated with color patches of the given geometry type and generating first metadata comprising a description of a location of the depth patch in the depth patch atlas; and generating second metadata comprising the associated constant depth value for other color patches; and
encoding in a data stream:
the generated color patch atlas;
the generated depth patch atlas; and
for each color patch, the information indicating whether the color picture is of the given geometry type, and, if so the first metadata, and if not, the second metadata.

5. The method of claim 4 , wherein a color data further comprises a description of at least one of a size or of a shape of at least one color patch in the color patch atlas.

6. The method of claim 4 , wherein the data stream comprises a sequence of color patch atlases, a sequence of depth patch atlases and associated metadata, each depth patch atlas and related metadata being associated with a color patch atlas, wherein the color patch atlases are structured by temporal information.

7. A device comprising electronic circuitry adapted for:
obtaining from a data stream:
a color patch atlas packing color pictures;
a depth patch atlas packing depth pictures corresponding to color patches of a given geometry type; and
metadata comprising, for each color patch of the color patch atlas:
de-projection data;
a color data comprising a description of a location of a color picture in the color image; and
an information indicating whether the color picture is of the given geometry type; and
if the information indicates that the color picture is of the given geometry type, a description of a location of a depth picture in the depth patch atlas; and
if the information indicates that the color picture is not of the given geometry type, a constant depth value; and

for each color patch of the given geometry type:
using de-projection data for de-projecting pixels of the color picture described by the color data at a depth value determined according to pixels of the corresponding depth patch in the depth atlas; and

for each other color patch:
using de-projection data for de-projecting pixels of the color picture described by the color data at the constant depth value in the associated metadata.

8. The device of claim 7 , wherein a color data further comprises a description of at least one of a size or of a shape of at least one color patch in the color patch atlas.

9. The device of claim 7 , wherein the data stream comprises a sequence of color patch atlases, a sequence of depth patch atlases and associated metadata, each depth patch atlas and related metadata being associated with a color patch atlas, wherein the color patch atlases are structured by temporal information.

10. A device comprising electronic circuitry adapted for:
obtaining a set of color patches, and, for each color patch, an information indicating whether the color patch is of a given geometry type, and if so, an associated depth patch, and if not, an associated constant depth value;
generating a color patch atlas by packing the color patches in a color patch atlas and generating a color data comprising a description of a location of each color patch as packed in the color patch atlas;
generating a depth patch atlas by packing depth patches associated with color patches of the given geometry type and generating first metadata comprising a description of a location of the depth patch in the depth patch atlas; and generating second metadata comprising the associated constant depth value for other color patches; and
encoding in a data stream:
the generated color patch atlas;
the generated depth patch atlas; and
for each color patch, the information indicating whether the color picture is of the given geometry type, and, if so the first metadata, and if not, the second metadata.

11. The device of claim 10 , wherein a color data further comprises a description of at least one of a size or of a shape of at least one color patch in the color patch atlas.

12. The device of claim 10 , wherein the data stream comprises a sequence of color patch atlases, a sequence of depth patch atlases and associated metadata, each depth patch atlas and related metadata being associated with a color patch atlas, wherein the color patch atlases are structured by temporal information.

13. A non-transitory computer readable medium having stored instructions which, when executed by one or more processors, cause the one or more processors to carry out the method of claim 1 .

14. A non-transitory computer readable medium having stored instructions which, when executed by one or more processors, cause the one or more processors to carry out the method of claim 4 .

Assignees

• Interdigital Vc Holdings Inc

Inventors

• Fleureau Julien
• Dore Renaud
• Thudor Franck

Classifications

• H04L65/75

  Media network packet handling · CPC title

• H04L65/756

  adapting media to device capabilities · CPC title

• H04L65/70

  Media network packetisation · CPC title

• G06T17/20

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G06T9/001 Primary

  Model-based coding, e.g. wire frame · CPC title