Method, an apparatus and a computer program product for spatial computing service session description for volumetric extended reality conversation

Patent metadata

Field	Value
Publication number	US-12375634-B2
Application number	US-202318151722-A
Country	US
Kind code	B2
Filing date	Jan 9, 2023
Priority date	Jan 21, 2022
Publication date	Jul 29, 2025
Grant date	Jul 29, 2025

Abstract

There is disclosed an apparatus and a method for spatial computing service session description for volumetric XR conversation. In accordance an embodiment the method comprises receiving volumetric video from a plurality of sources, the volumetric video data comprising color images and depth images representing at least a part of a scene; determining which of the plurality of sources belong to a same group; selecting a group identifier for the group; associating the volumetric video with the group identifier indicative of from which group the volumetric video data is received; associating the volumetric video with a source identifier indicative of from which source of the group the volumetric video data is received; associating the color image of the volumetric video with a color data identifier; and associating the depth image of the volumetric video with a depth data identifier.

First claim

The invention claimed is:

1. An apparatus for encoding comprising at least one processor, memory including computer program code, the memory and the computer program code configured to, with the at least one processor, cause the apparatus to:
receive volumetric video data from a plurality of sources, the volumetric video data comprising one or more color images and one or more depth images representing at least a part of a scene;
determine which of the plurality of sources belong to a same group;
select a group identifier for the group;
associate the volumetric video data with the group identifier indicative of from which group the volumetric video data is received;
associate the volumetric video data with a source identifier indicative of from which source of the group the volumetric video data is received;
associate the one or more color images of the volumetric video data with a color data identifier; and
associate the one or more depth images of the volumetric video data with a depth data identifier.

2. The apparatus according to claim 1 further caused to:
send the volumetric video data and the identifiers associated with the volumetric video data to a viewer device.

3. The apparatus according to claim 1 further caused to:
reproduce the volumetric video data; and
send the reproduced volumetric video data to a viewer device.

4. The apparatus according to claim 1 further caused to:
include the group identifier and the source identifier in a spatial computing service session description file.

5. The apparatus according to claim 4 further caused to:
obtain intrinsic and extrinsic parameters from the plurality of sources; and
include the intrinsic and extrinsic parameters in the spatial computing service session description file.

6. The apparatus according to claim 1 , wherein sources fixed with a same device rig are determined to belong to the same group.

7. The apparatus according to claim 1 further caused to:
initiate a call between two or more sources; and
create a spatial computing service session description file for relaying signaling information and metadata between the sources of the call.

8. The apparatus according to claim 7 further caused to:
segment multimedia information related to the volumetric video data from the plurality of sources into infrequent chunks and frequent chunks;
send the infrequent chunks at least during the initiation of the call; and
send the frequent chunks regularly during the call.

9. The apparatus according to claim 8 , wherein the infrequent chunks include one or more of the following:
a group index,
intrinsic parameters,
extrinsic parameters, and
video information; and
the frequent chunks include one or more of the following:
batch information specifying maximum number of frames that will be sent as a part of a video batch and including none or at least one of a local time stamp for the video frame, and an exposure time,
one or more frames of the volumetric video data, and audio data of the volumetric video data.

10. A method comprising:
receiving volumetric video data from a plurality of sources, the volumetric video data comprising one or more color images and one or more depth images representing at least a part of a scene;
determining which of the plurality of sources belong to a same group;
selecting a group identifier for the group;
associating the volumetric video data with the group identifier indicative of from which group the volumetric video data is received;
associating the volumetric video data with a source identifier indicative of from which source of the group the volumetric video data is received;
associating the one or more color images of the volumetric video data with a color data identifier; and
associating the one or more depth images of the volumetric video data with a depth data identifier.

11. The method according to claim 10 further comprising:
sending the volumetric video data and the identifiers associated with the volumetric video data to a viewer device.

12. The method according to claim 10 further comprising:
reproducing the volumetric video data; and
sending the reproduced volumetric video data to a viewer device.

13. The method according to claim 10 further comprising:
including the group identifier and the source identifier in a spatial computing service session description file.

14. The method according to claim 13 further comprising:
obtaining intrinsic and extrinsic parameters from the plurality of sources; and
including the intrinsic and extrinsic parameters in the spatial computing service session description file.

15. The method according to claim 10 , wherein sources fixed with a same device rig are determined to belong to the same group.

16. The method according to claim 10 further comprising:
initiating a call between two or more sources; and
creating a spatial computing service session description file for relaying signaling information and metadata between the sources of the call.

17. The method according to claim 16 further comprising:
segmenting multimedia information related to the volumetric video data from the plurality of sources into infrequent chunks and frequent chunks;
sending the infrequent chunks at least during the initiation of the call; and
sending the frequent chunks regularly during the call.

18. The method according to claim 17 , wherein:
the infrequent chunks include one or more of the following:
a group index,
intrinsic parameters,
extrinsic parameters, and
video information; and

the frequent chunks include one or more of the following:
batch information specifying maximum number of frames that will be sent as a part of a video batch and including none or at least one of a local time stamp for the video frame, and an exposure time,
one or more frames of the volumetric video data, and audio data of the volumetric video data.

Assignees

• Nokia Technologies Oy

Inventors

• Fasogbon Peter Oluwanisola
• Aksu Emre Baris
• Curcio Igor Danilo Diego
• You Yu
• Kondrad Lukasz
• Mate Sujeet Shyamsundar
• Ahsan Saba

Classifications

• H04L65/1069

  Session establishment or de-establishment · CPC title

• H04N2013/0092

  Image segmentation from stereoscopic image signals · CPC title

• H04N23/60

  Control of cameras or camera modules · CPC title

• H04L65/70

  Media network packetisation · CPC title

• H04N13/194 Primary

  Transmission of image signals · CPC title