Multi-atlas encapsulation of immersive media

Patent metadata

Field	Value
Publication number	US-12137225-B2
Application number	US-202217863049-A
Country	US
Kind code	B2
Filing date	Jul 12, 2022
Priority date	Nov 30, 2020
Publication date	Nov 5, 2024
Grant date	Nov 5, 2024

Abstract

Methods, apparatus, and systems that provide flexible encapsulation of volumetric video data in media files are disclosed. In one example aspect, a method for video processing includes receiving three-dimensional (3D) volumetric video data and encoding the three-dimensional volumetric video data into a media file. The 3D volumetric video data corresponds to one or more atlases, each comprising atlas data and one or more two-dimensional (2D) components. The atlas data and the one or more 2D components are stored in one or more media tracks in the media file.

First claim

What is claimed is:

1. A method for video processing, comprising:
receiving three-dimensional (3D) volumetric video data; and
encoding the 3D volumetric video data into a media file according to a file format,
wherein the 3D volumetric video data corresponds to atlases, each atlas comprising atlas data and one or more two-dimensional (2D) components, and wherein the file format specifies that the atlas data and the one or more 2D components of each atlas are stored in one or more media tracks in the media file, wherein the file format further specifies at least one 2D component of the one or more 2D components that corresponds to an atlas is stored in a component track of the one or more media tracks, and wherein a part of the atlas data that the at least one 2D component corresponds to and is not common to all of the atlases is stored in the component track.

2. The method of claim 1 , wherein the one or more 2D components for each atlas comprise at least a geometry component, an occupancy component, or an attribute component.

3. The method of claim 1 , wherein the atlas data comprises projection relationship of projecting the 3D volumetric video data from a 3D space to a 2D plane.

4. The method of claim 1 , wherein the one or more media tracks includes a volumetric visual track with a specific sample entry type, the volumetric visual track comprising parameter information common to all of the atlases.

5. The method of claim 1 , wherein the one or more media tracks are organized into multiple groups, each group corresponding to the atlases arranged according to a grouping criterion associated with the 3D volumetric video data.

6. A method for video processing, comprising:
decoding a media file that represents three-dimensional (3D) volumetric video data according to a file format,
wherein the 3D volumetric video data corresponds to atlases, each atlas comprising atlas data and one or more two-dimensional (2D) components, and wherein the file format specifies that the atlas data and the one or more 2D components of each atlas are stored in one or more media tracks in the media file, wherein the file format further specifies at least one 2D component of the one or more 2D components that corresponds to an atlas is stored in a component track of the one or more media tracks, and wherein a part of the atlas data that the at least one 2D component corresponds to and is not common to all of the atlases is stored in the component track; and
reconstructing the 3D volumetric video data based on at least one media track of the one or more media tracks.

7. The method of claim 6 , wherein the one or more 2D components for each atlas comprise at least a geometry component, an occupancy component, or an attribute component.

8. The method of claim 6 , wherein the atlas data comprises projection relationship of projecting the 3D volumetric video data from a 3D space to a 2D plane.

9. The method of claim 6 , wherein the one or more media tracks includes a volumetric visual track with a specific sample entry type, the volumetric visual track comprising parameter information common to all of the atlases.

10. The method of claim 6 , wherein the one or more media tracks are organized into multiple groups, each group corresponding to the atlases arranged according to a grouping criterion associated with the 3D volumetric video data.

11. A video processing apparatus comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
receive three-dimensional (3D) volumetric video data; and
encode the 3D volumetric video data into a media file according to a file format,
wherein the 3D volumetric video data corresponds to atlases, each atlas comprising atlas data and one or more two-dimensional (2D) components, and wherein the file format specifies that the atlas data and the one or more 2D components of each atlas are stored in one or more media tracks in the media file, wherein the file format further specifies at least one 2D component of the one or more 2D components that corresponds to an atlas is stored in a component track of the one or more media tracks, and wherein a part of the atlas data that the at least one 2D component corresponds to and is not common to all of the atlases is stored in the component track.

12. The apparatus of claim 11 , wherein the one or more 2D components for each atlas comprise at least a geometry component, an occupancy component, or an attribute component.

13. The apparatus of claim 11 , wherein the atlas data comprises projection relationship of projecting the 3D volumetric video data from a 3D space to a 2D plane.

14. The apparatus of claim 11 , wherein the one or more media tracks includes a volumetric visual track with a specific sample entry type, the volumetric visual track comprising parameter information common to all of the atlases.

15. The apparatus of claim 11 , wherein the one or more media tracks are organized into multiple groups, each group corresponding to the atlases arranged according to a grouping criterion associated with the 3D volumetric video data.

16. A video processing apparatus comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
decode a media file that represents three-dimensional (3D) volumetric video data according to a file format,
wherein the 3D volumetric video data corresponds to atlases, each atlas comprising atlas data and one or more two-dimensional (2D) components, and wherein the file format specifies that the atlas data and the one or more 2D components of each atlas are stored in one or more media tracks in the media file, wherein the file format further specifies at least one 2D component of the one or more 2D components that corresponds to an atlas is stored in a component track of the one or more media tracks, and wherein a part of the atlas data that the at least one 2D component corresponds to and is not common to all of the atlases is stored in the component track; and
reconstruct the 3D volumetric video data based on at least one media track of the one or more media tracks.

17. The apparatus of claim 16 , wherein the one or more 2D components for each atlas comprise at least a geometry component, an occupancy component, or an attribute component.

18. The apparatus of claim 16 , wherein the atlas data comprises projection relationship of projecting the 3D volumetric video data from a 3D space to a 2D plane.

19. The apparatus of claim 16 , wherein the one or more media tracks includes a volumetric visual track with a specific sample entry type, the volumetric visual track comprising parameter information common to all of the atlases.

20. The apparatus of claim 16 , wherein the one or more media tracks are organized into multiple groups, each group corresponding to the atlases arranged according to a grouping criterion associated with the 3D volumetric video data.

Assignees

• Zte Corp

Inventors

• Li Qiuting

Classifications

• H04N19/46

  Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) · CPC title

• H04N19/423

  characterised by memory arrangements (H04N19/433 takes precedence) · CPC title

• H04N19/132

  Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking · CPC title

• H04N19/136 Primary

  Incoming video signal characteristics or properties · CPC title

• H04N19/597 Primary

  specially adapted for multi-view video sequence encoding · CPC title