Content partition based real time communication for immersive media

Patent metadata

Field	Value
Publication number	US-12407745-B2
Application number	US-202118575161-A
Country	US
Kind code	B2
Filing date	Dec 23, 2021
Priority date	Nov 10, 2021
Publication date	Sep 2, 2025
Grant date	Sep 2, 2025

Abstract

Systems and techniques for selective transport of spatially partitioned immersive media content are described herein. In an example, the system may negotiate an attribute for media content between a media producer and a media consumer. A content structure for the media content may be determined based on the negotiation of the attribute, and a network packet may be created with information corresponding to the determined content structure encoded into a transport protocol header of the network packet. The network packet may be filtered to produce filtered media content. The filtered media content may be transmitted to the media consumer.

First claim

What is claimed is:

1. A system for selective transport of spatially partitioned immersive media content comprising:
at least one processor; and
memory including instructions that, when executed by the at least one processor, cause the at least one processor to perform operations to:
negotiate, via a signaling service, an attribute for media content between a media producer and a media consumer;
determine, based on the negotiating, a content structure for the media content;
create a network packet, the network packet including information corresponding to the determined content structure that is encoded into a transport protocol header of the network packet;
filter the network packet using partition identifiers received from the media consumer to produce filtered media content; and
transmit the filtered media content to the media consumer.

2. The system of claim 1 , wherein the partition identifiers may describe a region of the media content, a segment of the media content, a spatial partition of the media content, a tile of the media content, or a slice of the media content.

3. The system of claim 1 , wherein the instructions to negotiate the attribute for the media content between the media producer and the media consumer further comprises instructions that, when executed by the at least one processor, cause the at least one processor to perform operations to:
initiate a session description protocol (SDP) communication session between the media producer and the media consumer via the signaling service;
identify a codec that is compatible with the media producer and the media consumer;
identify a content partition layout that is compatible with the media producer and the media consumer; and
transmit information for the codec and the content partition layout to the media producer and the media consumer via the SDP communication session.

4. The system of claim 1 , wherein the content structure includes regions of the media content or tiles that represent the regions of the media content.

5. The system of claim 4 , wherein the transport protocol header includes identifiers for the regions of the media content or the tiles that represent the regions of the media content.

6. The system of claim 1 , wherein the network packet is a real-time transport protocol (RTP) packet.

7. The system of claim 1 , wherein the transport protocol header includes a partition descriptor, the partition descriptor including a partition identifier, a type for a partition of the media content, and a priority level for the partition.

8. The system of claim 1 , wherein the transport protocol header includes a partition structure descriptor, the partition structure descriptor including a first axis offset, a second axis offset, a width, and a height of a partition of the media content.

9. The system of claim 1 , the memory further comprising instructions that, when executed by the at least one processor, cause the at least one processor to perform operations to:
extract features from the media content based on the content structure; and
encode the features for inclusion in the network packet.

10. The system of claim 9 , wherein the instructions to transmit the filtered media content to the media consumer further comprises instructions that, when executed by the at least one processor, cause the at least one processor to perform operations to:
multiplex encoded video content and the encoded features of the filter media content into an encoded bitstream; and
transmit the encoded bitstream to the media consumer.

11. The system of claim 10 , wherein the encoded features are used for automated interpretation of the media content by a computing device.

12. The system of claim 1 , wherein the instruction to filter the network packet to produce the filtered media content further comprises instructions that, when executed by the at least one processor, cause the at least one processor to perform operations to:
select a partition of the media content using the attribute and the content structure; and
retrieve a media fragment for a portion of the media content associated with the partition, wherein the filtered media content includes the media fragment for the portion of the media content.

13. The system of claim 1 , wherein the packet is filtered by a network relay server that is provided by a server device, a virtual network function, a microservice, or a software component executing on an edge node in communication with an edge computing network.

14. The system of claim 1 , wherein the media content is video content according to a motion picture experts group (MPEG) format.

15. The system of claim 1 , wherein the spatially partitioned immersive media content is three-dimensional video content, virtual reality content, augmented reality content, or perception enhanced two-dimensional video content.

16. At least one non-transitory machine-readable medium including instructions for selective transport of spatially partitioned immersive media content that, when executed by at least one processor, cause the at least one processor to perform operations to:
negotiate, via a signaling service, an attribute for media content between a media producer and a media consumer;
determine, based on the negotiating, a content structure for the media content;
create a network packet, the network packet including information corresponding to the determined content structure that is encoded into a transport protocol header of the network packet;
filter the network packet using partition identifiers received from the media consumer to produce filtered media content; and
transmit the filtered media content to the media consumer.

17. The at least one non-transitory machine-readable medium of claim 16 , wherein the instructions to negotiate the attribute for the media content between the media producer and the media consumer further comprises instructions that, when executed by the at least one processor, cause the at least one processor to perform operations to:
initiate a session description protocol (SDP) communication session between the media producer and the media consumer via the signaling service;
identify a codec that is compatible with the media producer and the media consumer;
identify a content partition layout that is compatible with the media producer and the media consumer; and
transmit information for the codec and the content partition layout to the media producer and the media consumer via the SDP communication session.

18. The at least one non-transitory machine-readable medium of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the at least one processor to perform operations to:
extract features from the media content based on the content structure; and
encode the features for inclusion in the network packet.

19. A method for selective transport of spatially partitioned immersive media content comprising:
negotiating, via a signaling service, an attribute for media content between a media producer and a media consumer;
determining, based on the negotiating, a content structure for the media content;
creating a network packet, the network packet including information corresponding to the determined content structure that is encoded into a transport protocol header of the network packet;
filtering the network packet using partition identifiers received from the media consumer to produce filtered media content; and
transmitting the filtered media content to the media consumer.

Assignees

• Intel Corp

Inventors

• Shen Gang
• Moustafa Hassnaa
• Dai Jianhui
• Wu Qiujiao
• Qiu Jianlin

Classifications

• H04N21/845

  Structuring of content, e.g. decomposing content into time segments · CPC title

• H04N21/6437 Primary

  Real-time Transport Protocol [RTP] · CPC title

• H04L65/1069

  Session establishment or de-establishment · CPC title

• H04L65/612

  for unicast · CPC title

• H04L65/80

  Responding to QoS · CPC title