XR streaming system for lightweight XR device and operation method therefor

Patent metadata

Field	Value
Publication number	US-12501094-B2
Application number	US-202318529607-A
Country	US
Kind code	B2
Filing date	Dec 5, 2023
Priority date	Oct 27, 2021
Publication date	Dec 16, 2025
Grant date	Dec 16, 2025

Abstract

Proposed are a mediation server operation method for an XR streaming service and XR streaming system supporting same, the mediation server operation method comprising the steps of: if a mediation server receives an XR content playback request from a connected user device, searching for an idle split rendering server; allocating the found idle split rendering server to the user device; receiving image data and sensor data from the user device; generating a rotation matrix on the basis of the received image data and sensor data; and transmitting the rotation matrix to the allocated split rendering server.

First claim

What is claimed is:

1 . A cross reality (XR) streaming system comprising:
a user device configured to collect and transmit video data and sensor data; a mediation server configured to:
allocate a split rendering server to the user device in response to an XR content request from the user device,
configure a rotation matrix based on the video data and the sensor data received from the user device, and
deliver the rotation matrix to the split rendering server allocated to the user device; and
the allocated split rendering server configured to generate a 2D video by performing 3D rendering based on the rotation matrix received from the mediation server, and transmitting the generated 2D video to the user device.

2 . The XR streaming system of claim 1 , wherein the mediation server is configured to receive the sensor data related to a location of the user device and a pose of the user device.

3 . The XR streaming system of claim 1 , wherein the mediation server is configured to manage split rendering servers, search for an idle split rendering server in response to the XR content request from the user device, and allocate the idle split rendering server to the user device.

4 . The XR streaming system of claim 1 , wherein in response to there being no idle split rendering server, the mediation server is configured to request the user device to wait for split rendering server allocation.

5 . The XR streaming system of claim 1 , wherein the mediation server is configured to perform transmission control protocol (TCP)-based communication with the user device to determine whether the user device supports a quick user datagram protocol (UDP) Internet Connection (QUIC) communication protocol.

6 . The XR streaming system of claim 1 , wherein the mediation server is configured to perform user datagram protocol (UDP)-based data transmission and reception with the split rendering server.

7 . An operation method of a mediation server supporting a cross reality (XR) streaming service, the method comprising:
in response to the mediation server receiving an XR content playback request from a connected user device, searching for an idle split rendering server; allocating the found idle split rendering server to the user device; receiving video data and sensor data from the user device; generating a rotation matrix based on the received video data and the received sensor data; and transmitting the rotation matrix to the allocated split rendering server.

8 . The method of claim 7 , further comprising:
in response to there being no idle split rendering server, requesting the user device to wait for split rendering server allocation.

9 . The method of claim 7 , further comprising:
performing transmission control protocol (TCP)-based communication with the user device to determine whether the user device supports a quick user datagram protocol (UDP) Internet Connection (QUIC) communication protocol.

10 . The method of claim 7 , wherein the transmitting includes performing user datagram protocol (UDP)-based data transmission and reception with the split rendering server.

Assignees

• Korea Electronics Technology

Inventors

• Jang Junhwan
• Park Woochool
• Choi Minsu
• Lee Junsuk
• Koo Bonjae

Classifications

• H04N21/8146 Primary

  involving graphical data, e.g. 3D object, 2D graphics · CPC title

• H04N21/44209

  Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network (arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896) · CPC title

• H04N21/25841

  involving the geographical location of the client (retrieval from the Internet by querying based on geographical locations G06F16/9537; arrangements for identifying locations of receiving stations in broadcast systems H04H60/51; location of the user terminal in data switching networks H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00) · CPC title

• H04N21/234

  Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title