Video-based point cloud streams

Patent metadata

Field	Value
Publication number	US-12238394-B2
Application number	US-202017763346-A
Country	US
Kind code	B2
Filing date	Sep 23, 2020
Priority date	Sep 27, 2019
Publication date	Feb 25, 2025
Grant date	Feb 25, 2025

Abstract

Systems, methods, and instrumentalities are disclosed that relate to the processing of a media container file associated with 3D video data. The media container file may indicate that certain video-based point cloud compression (V-PCC) component tracks may be played together as a playout group. These V-PCG component tracks may represent respective encoded versions of one or more V-PCC components, and a video decoding device may play the tracks together in response to determining that the tracks belong to the same playout track group. The video decoding device may also determine from the media container file that certain PCC component tracks include tile groups that correspond to different objects in a point cloud or different parts of a same object in the point cloud. The video decoding device may decode these tile groups independently from each other so that a subset of the objects or parts of the point cloud may be accessed without also accessing the rest of the objects or parts.

First claim

The invention claimed is:

1. A video decoding device, comprising:
a processor configured to:
obtain a media container file, wherein the media container file indicates a first playout track group and a grouping of a first video-based point cloud compression (V-PCC) component track and a second V-PCC component track into the first playout track group, wherein the first V-PCC component track is associated with a first V-PCC component and the second V-PCC component track is associated with a second V-PCC component, and wherein the media container file further indicates that the first V-PCC component track and the second V-PCC component track belong to different alternate groups associated with two different V-PCC components;
determine, based on the media container file, that the first V-PCC component track and the second V-PCC component track are to be played together; and
based on the determination that the first V-PCC component track and the second V-PCC component track are to be played together:
decode the first V-PCC component track and the second V-PCC component track; and
play the decoded first V-PCC component track and the decoded second V-PCC component track together.

2. The video decoding device of claim 1 , wherein the media container file includes an identifier of the first playout track group, and wherein the processor being configured to determine that the first V-PCC component track and the second V-PCC component track are to be played together comprises the processor being configured to determine that the first V-PCC component track and the second V-PCC component track are associated with the identifier of the first playout track group.

3. The video decoding device of claim 1 , wherein the media container file defines the first playout track group as an extension of a track group.

4. The video decoding device of claim 1 , wherein the media container file further indicates that the first V-PCC component track and a third V-PCC component track are associated with a same V-PCC component and that the third V-PCC component track is grouped into a second playout track group.

5. The video decoding device of claim 4 , wherein the first V-PCC component track and the second V-PCC component track are associated with a same encoded version and wherein the first V-PCC component track and the third V-PCC component track are associated with different encoded versions.

6. The video decoding device of claim 4 , wherein the media container file further indicates that the second V-PCC component track and a fourth V-PCC component track are associated with a same V-PCC component and that the fourth V-PCC component track is grouped into the second playout track group.

7. The video decoding device of claim 6 , wherein the processor is further configured to decode the third V-PCC component track and the fourth V-PCC component track and play the decoded third V-PCC component track and the decoded fourth V-PCC component track together.

8. The video decoding device of claim 1 , wherein the media container file further indicates the first V-PCC component track also belongs to a second playout track group.

9. The video decoding device of claim 1 , wherein the processor is further configured to determine, based on the media container file, that at least one of the first V-PCC component track or the second V-PCC component track comprises tile groups that correspond to different objects in a point cloud or to different parts of an object in the point cloud.

10. The video decoding device of claim 9 , wherein the processor is further configured to decode the tile groups independently from each other.

11. A method for video decoding, the method comprising:
obtaining a media container file, wherein the media container file indicates a first playout track group and a grouping of a first video-based point cloud compression (V-PCC) component track and a second V-PCC component track into the first playout track group, and wherein the media container file further indicates that the first V-PCC component track and the second V-PCC component track belong to different alternate groups associated with two different V-PCC components;
determining, based on the media container file, that the first V-PCC component track and the second V-PCC component track are to be played together; and
in response to determining that the first V-PCC component track and the second V-PCC component track are to be played together:
decoding the first V-PCC component track and the second V-PCC component track; and
playing the decoded first V-PCC component track and the decoded second V-PCC component track together.

12. The method of claim 11 , wherein the media container file includes an identifier of the first playout track group, and wherein determining that the first V-PCC component track and the second V-PCC component track are to be played together comprises determining that the first V-PCC component track and the second V-PCC component track are associated with the identifier of the first playout track group.

13. The method of claim 11 , wherein the media container file defines the first playout track group as an extension of a track group.

14. The method of claim 11 , wherein the media container file further indicates that the first V-PCC component track and a third V-PCC component track are associated with a same V-PCC component and that the third V-PCC component track is grouped into a second playout track group.

15. The method of claim 14 , wherein the first V-PCC component track and the second V-PCC component track are associated with a same encoded version and wherein the first V-PCC component track and the third V-PCC component track are associated with different encoded versions.

16. The method of claim 15 , wherein the media container file further indicates that the second V-PCC component track and a fourth V-PCC component track are associated with a same V-PCC component and that the fourth V-PCC component track is grouped into the second playout track group.

17. The method of claim 16 , further comprising decoding the third V-PCC component track and the fourth V-PCC component track and playing the decoded third V-PCC component track and the decoded fourth V-PCC component track together.

18. The method of claim 11 , wherein the media container file further indicates that the first V-PCC component track also belongs to a second playout track group.

19. The method of claim 11 , further comprising determining, based on the media container file, that at least one of the first V-PCC component track or the second V-PCC component track comprises tile groups that correspond to different objects in a point cloud or to different parts of an object in the point cloud.

20. A video encoding device, comprising:
a processor configured to:
determine that multiple video-based point cloud compression (V-PCC) component tracks are associated with a same encoding version and are to be played together;
indicate, in a media container file, that the multiple V-PCC component tracks belong to a same playout track group, but to different alternate groups that are associated with different V-PCC components, wherein each of the alternate groups includes different encoded versions of a V-PCC component track; and
provide the media container file to a receiving device.

Assignees

• Vid Scale Inc
• Interdigital Madison Patent Holdings Sas

Inventors

• Hamza Ahmed
• He Yong

Classifications

• H04N19/174

  the region being a slice, e.g. a line of blocks or a group of blocks · CPC title

• H04N19/597

  specially adapted for multi-view video sequence encoding · CPC title

• H04N21/85406 Primary

  involving a specific file format, e.g. MP4 format · CPC title

• H04N21/816 Primary

  involving special video data, e.g 3D video · CPC title

• H04N21/440245

  the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment · CPC title