Different atlas packings for volumetric video

Patent metadata

Field	Value
Publication number	US-12212784-B2
Application number	US-202117916367-A
Country	US
Kind code	B2
Filing date	Apr 1, 2021
Priority date	Apr 7, 2020
Publication date	Jan 28, 2025
Grant date	Jan 28, 2025

Abstract

Methods, devices and stream are disclosed to encode and decode a scene (such as a point cloud) in the context of a patch-based transmission of a volumetric video content. Attributes of points of the scene are projected onto patches. Every point has a geometry attribute. For other attributes, like transparency of displacement attribute, some points may have no value. According to the present principles, each attribute is encoded in a different atlas with its own layout. This allow to save pixel rate in memory of the renderer.

First claim

The invention claimed is:

1. A method comprising obtaining, from a data stream, a first image packing first patches, a second image packing second patches, and associated metadata, the metadata comprising a list of data items, wherein a data item comprises:
a first location and a size of a region of the first image corresponding to a current first patch, wherein the first patch encodes a projection of a part of points of a 3D scene;
an information indicating whether a second patch corresponding to the current first patch is present in the second image; and
on a condition that the information indicates the presence of the second patch in the second image, the metadata comprises a second location of a region of the second image corresponding to the second patch;
the method further comprising, for a data item of the list of patch data items:
decoding the first patch at the first location and of the size in the first image; and
on a condition that the information indicates the presence of the second patch in the second image, decoding the second patch at the second location and of the size in the second image.

2. The method of claim 1 , wherein the first patch present in the first image encodes a first attribute of a part of points of the 3D scene onto patches and if the second patch is present in the second image, the second patch encodes a second attribute of the part of points of the 3D scene, the first attribute being different from the second attribute.

3. The method of claim 1 , wherein the data item comprises projection information for the current first patch and wherein first patches of the first image encode a geometry attribute, the method further comprising:
generating a 3D scene according to decoded first and second patches.

4. A device comprising a processor configured for decoding, from a data stream, a first image packing first patches, a second image packing second patches, and associated metadata, the metadata comprising a list of data items, wherein a data item comprises:
a first location and a size of a region of the first image corresponding to a current first patch, wherein a first patch encodes a projection of a part of points of a 3D scene;
an information indicating whether a second patch corresponding to the current first patch is present in the second image; and
on a condition that the information indicates the presence of the second patch in the second image, the metadata comprises a second location of a region of the second image corresponding to the second patch;
the processor further configured for, for a data item of the list of patch data items:
decoding a first patch at the first location and of the size in the first image; and
on a condition that the information indicates the presence of the second patch in the second image, decoding the second patch at the second location and of the size in the second image.

5. The device of claim 4 , wherein the first patch present in the first image encodes a first attribute of a part of points of the 3D scene onto patches and if the second patch is present in the second image, the second patch encodes a second attribute of the part of points of the 3D scene, the first attribute being different from the second attribute.

6. The device of claim 4 , wherein the data item comprises projection information for the current patch and wherein patches of the first image encode a geometry attribute, the processor being further configured for generating a 3D scene according to decoded first and second patches.

7. A method comprising:
obtaining a set of first and second patches, a first patch encoding a projection of a first attribute of a part of points of a 3D scene, a second patch encoding a projection of the first attribute and a second attribute of the part of points of the 3D scene;
packing first patches in a first image and packing second patches in a second image; and
generating a data stream comprising the first image, the second image, and associated metadata, wherein the metadata comprises, for a current first patch:
a first location and a size of a region of the first image corresponding to the current first patch;
an information indicating whether a second patch corresponds to the current first patch; and
if so, adding a second location of a region of the second image corresponding to the second patch to the metadata.

8. The method of claim 7 , wherein the first attribute is different from the second attribute.

9. The method of claim 7 , wherein the first attribute is a geometry attribute.

10. A device comprising a processor configured for:
obtaining a set of first and second patches, a first patch encoding a projection of a first attribute of a part of points of a 3D scene, a second patch encoding a projection of the first attribute and a second attribute of the part of points of the 3D scene;
packing first patches in a first image and packing second patches of in a second image; and
generating a data stream comprising the first image, the second image, and associated metadata, wherein the metadata comprises, for a current first patch:
a first location and a size of a region of the first image corresponding to the current first patch;
an information indicating whether a second patch corresponds to the current first patch; and
if so, adding a second location of a region of the second image corresponding to the second patch to the metadata.

11. The device of claim 10 , wherein the first attribute is different from the second attribute.

12. The device of claim 10 , wherein the first attribute is a geometry attribute.

13. A non-transitory computer readable storage medium storing instructions, which, when executed by one or more processors, perform the method of claim 1 .

14. A non-transitory computer readable storage medium storing instructions, which, when executed by one or more processors, perform the method of claim 7 .

15. A non-transitory computer readable storage medium storing:
a first image packing first patches;
a second image packing second patches; and
associated metadata comprising a list of data items, wherein a data item comprises:
a first location and a size of a region of the first image corresponding to a current first patch, wherein the first patch encodes a projection of a part of points of a 3D scene;
an information indicating whether a second patch corresponding to the current first patch is present in the second image; and
on a condition that the information indicates the presence of the second patch in the second image, the metadata comprises a second location of a region of the second image corresponding to the second patch.

16. The non-transitory computer readable storage medium of claim 15 , wherein the first patch present in the first image encodes a first attribute of a part of points of the 3D scene onto patches and if the second patch is present in the second image, the second patch encodes a second attribute of the part of points of the 3D scene.

17. The non-transitory computer readable storage medium of claim 16 , wherein the first attribute is different from the second attribute.

18. The non-transitory computer readable storage medium of claim 16 , wherein the first attribute is a geometry attribute.

Assignees

• Interdigital Ce Patent Holdings Sas

Inventors

• Chupeau Bertrand
• Dore Renaud
• Thudor Franck

Classifications

• G06T3/16

  Spatio-temporal transformations, e.g. video cubism · CPC title

• H04N19/46

  Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) · CPC title

• H04N19/597

  specially adapted for multi-view video sequence encoding · CPC title

• H04N21/2353

  specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata · CPC title

• H04N21/85406

  involving a specific file format, e.g. MP4 format · CPC title