Rendering workload management for extended reality

Patent metadata

Field	Value
Publication number	US-2023215074-A1
Application number	US-202217567793-A
Country	US
Kind code	A1
Filing date	Jan 3, 2022
Priority date	Jan 3, 2022
Publication date	Jul 6, 2023
Grant date	—

Abstract

A method by a computing system of a device includes receiving a request to render a frame comprising one or more virtual content and determining, for the one or more virtual content, an associated characteristic. The method further includes determining whether to reduce a rendering workload associated with rendering the frame to satisfy one or more power or thermal constraints associated with the device. In response to a determination to reduce the rendering workload, the method further includes generating a set of rending parameters for rendering the frame in order to reduce the rendering workload. At least one rendering parameter in the set of rendering parameters is determined based on the characteristic associated with the one or more virtual content. The method thus includes rendering the frame in accordance with the set of rending parameters so as to satisfy the one or more power or thermal constraints.

First claim

What is claimed is:

1 . A method comprising, by a computing system of a device:
receiving a request to render a frame comprising one or more virtual content; determining, for each of the one or more virtual content, an associated characteristic; determining whether to reduce a rendering workload associated with rendering the frame to satisfy one or more power or thermal constraints associated with the device; in response to a determination to reduce the rendering workload, generating a set of rending parameters for rendering the frame in order to reduce the rendering workload, wherein at least one rendering parameter in the set of rendering parameters is determined based on the characteristic associated with at least one of the one or more virtual content; and rendering the frame in accordance with the set of rending parameters so as to satisfy the one or more power or thermal constraints.

2 . The method of claim 1 , wherein receiving the request to render the frame comprises receiving the request from a second device communicatively coupled to the device.

3 . The method of claim 1 , wherein determining, for each of the one or more virtual content, an associated characteristic comprises determining one or more of a foveal region, an object dimension, or a viewing distance.

4 . The method of claim 1 , wherein generating the set of rending parameters comprises generating one or more of an altered viewport, an altered frame rate, an altered resolution, an altered bit depth, an altered one or more color channels, an altered pose update threshold, an altered depth continuity, an altered content range, an altered depth density, an altered near-field depth, an altered far-field depth, an altered brightness, an altered contrast, or an altered tone.

5 . The method of claim 1 , further comprising:
subsequent to receiving the request to render the frame: generating a prediction of a duration for rendering the frame based on a current rendering workload of the device and the one or more power or thermal constraints associated with the device; selecting one of a plurality of predetermined rendering workload modes based on the prediction of the duration for rendering the frame; and rendering the frame in accordance with the selected one of the plurality of predetermined rendering workload modes so as to satisfy the one or more power or thermal constraints.

6 . The method of claim 5 , wherein the plurality of predetermined rendering workload modes comprises a high performance rendering workload mode, a medium performance workload processing mode, and a low performance rendering workload mode.

7 . The method of claim 1 , wherein the device comprises one or more first processors and a second device comprises one or more second processors, the device being communicatively coupled to the second device, the method further comprising:
determining a rendering workload associated with rendering the frame to satisfy the one or more power or thermal constraints associated with the device and a target quality of service (QoS) associated with the device and the second device; and dynamically switching between rendering the frame utilizing the one or more first processors and rendering the frame utilizing the one or more second processors based on the one or more power or thermal constraints and the target QoS.

8 . A device, comprising:
a non-transitory computer-readable storage medium including instructions; and one or more processors coupled to the non-transitory computer-readable storage medium, the one or more processors configured to execute the instructions to:
receive a request to render a frame comprising one or more virtual content;
determine, for each of the one or more virtual content, an associated characteristic;
determine whether to reduce a rendering workload associated with rendering the frame to satisfy one or more power or thermal constraints associated with the first device;
in response to a determination to reduce the rendering workload, generate a set of rending parameters for rendering the frame in order to reduce the rendering workload, wherein at least one rendering parameter in the set of rendering parameters is determined based on the characteristic associated with at least one of the one or more virtual content; and
render the frame in accordance with the set of rending parameters so as to satisfy the one or more power or thermal constraints.

9 . The device of claim 8 , wherein the instructions to receive the request to render the frame further comprises instructions to receive the request from a second device communicatively coupled to the device.

10 . The device of claim 8 , wherein the instructions to determine, for each of the one or more virtual content, the associated characteristic further comprises instructions to determine one or more of a foveal region, an object dimension, or a viewing distance.

11 . The device of claim 8 , wherein the instructions to generate the set of rending parameters comprises further comprises instructions to generate one or more of an altered viewport, an altered frame rate, an altered resolution, an altered bit depth, an altered one or more color channels, an altered pose update threshold, an altered depth continuity, an altered content range, an altered depth density, an altered near-field depth, an altered far-field depth, an altered brightness, an altered contrast, or an altered tone.

12 . The device of claim 8 , wherein the instructions further comprises instructions to:
subsequent to receiving the request to render the frame:
generate a prediction of a duration for rendering the frame based on a current rendering workload of the first device and the one or more power or thermal constraints associated with the first device;
select one of a plurality of predetermined rendering workload modes based on the prediction of the duration for rendering the frame; and
render the frame in accordance with the selected one of the plurality of predetermined rendering workload modes so as to satisfy the one or more power or thermal constraints.

13 . The device of claim 12 , wherein the plurality of predetermined rendering workload modes comprises a high performance rendering workload mode, a medium performance workload processing mode, and a low performance rendering workload mode.

14 . The device of claim 8 , wherein the device comprises one or more first processors and a second device comprises one or more second processors, the device being communicatively coupled to the second device, the instructions further comprising instructions to:
determine a rendering workload associated with rendering the frame to satisfy the one or more power or thermal constraints associated with the device and a target quality of service (QoS) associated with the device and the second device; and dynamically switch between rendering the frame utilizing the one or more first processors and rendering the frame utilizing the one or more second processors based on the one or more power or thermal constraints and the target QoS.

15 . A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of a device, cause the one or more processors to:
receive a request to render a frame comprising one or more virtual content; determining, for each of the one or more virtual content, an associated characteristic; determine whether to reduce a rendering workload associated with rendering the frame to satisfy one or more power or thermal constraints associated with the device; in response to a determination to reduce

Assignees

• Meta Platforms Tech Llc

Inventors

• Gorbatov Eugene

Classifications

• G06T15/005 Primary

  General purpose rendering architectures · CPC title

• G06F1/3243 Primary

  Power saving in microcontroller unit · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title