Compensating for high head movement in head-mounted displays

Patent metadata

Field	Value
Publication number	US-11650658-B2
Application number	US-202016868652-A
Country	US
Kind code	B2
Filing date	May 7, 2020
Priority date	Apr 24, 2017
Publication date	May 16, 2023
Grant date	May 16, 2023

Abstract

When the speed of head movement exceeds the processing capability of the system, a reduced depiction is displayed. As one example, the resolution may be reduced using coarse pixel shading in order to create a new depiction at the speed of head movement. In accordance with another embodiment, only the region the user is looking at is processed in full resolution and the remainder of the depiction is processed at lower resolution. In still another embodiment, the background depictions may be blurred or grayed out to reduce processing time.

First claim

What is claimed is:

1. A non-transitory storage medium comprising instructions that when executed cause a graphics processor comprising a plurality of graphics processing cores to:
receive, in the graphics processor, a first workload prepared by a virtual reality application when a rate of movement of a user's head exceeds a threshold level, and receive a second workload prepared by the virtual reality application when the rate of movement does not exceed the threshold level;
render the first workload in the graphics processor, comprising to render, from the first workload, a first plurality of frames having a reduced pixel density in a rendering engine of the graphics processor, including to pixel shade the first workload at a first shading rate;
provide the first plurality of frames to a display engine of the graphics processor;
thereafter send the first plurality of frames to a display panel;
render the second workload in the graphics processor, comprising to render, from the second workload, a second plurality of frames in the rendering engine, the rendering engine to render at least some of the second plurality of frames comprising separate surfaces, including to pixel shade the second workload at a second shading rate, the second shading rate is twice the first shading rate;
provide the second plurality of frames to the display engine;
combine the separate surfaces of the at least some of the second plurality of frames in the display engine; and
thereafter send the second plurality of frames to the display panel.

2. The non-transitory storage medium of claim 1 , further comprising instructions that when executed cause the graphics processor to pixel shade the first workload at a lower resolution than the second workload.

3. The non-transitory storage medium of claim 1 , further comprising instructions that when executed cause the graphics processor to:
render a first region of a first frame of the first plurality of frames at a first resolution; and
render a second region of the first frame at a second resolution lower than the first resolution, the first region comprising a foveated region.

4. The non-transitory storage medium of claim 3 , further comprising instructions that when executed cause the graphics processor to separately send the first region comprising a foreground surface and the second region comprising a background surface to the display panel, wherein the display panel comprises a head-mounted display having a first buffer to store the foreground surface and a second buffer to store the background surface.

5. The non-transitory storage medium of claim 1 , further comprising instructions that when executed cause the graphics processor to render the second plurality of frames having a pixel density greater than the reduced pixel density.

6. The non-transitory storage medium of claim 1 , further comprising instructions that when executed cause the graphics processor to receive, in the graphics processor, the first workload from the virtual reality application when the rate of movement of the user's head exceeds the threshold level, the threshold level comprising a processing capability of the graphics processor.

7. The non-transitory storage medium of claim 1 , further comprising instructions that when executed cause the graphics processor to render the first plurality of frames only in a foveated region.

8. A graphics processing unit, comprising:
an interface to couple the graphics processing unit to a processor,
a plurality of texture units,
a shared memory coupled to the plurality of texture units,
a plurality of register files coupled to the shared memory,
a plurality of load/store units coupled to the shared memory, and
a plurality of graphics processing cores coupled to the plurality of register files, and
a non-transitory storage medium comprising instructions that when executed cause at least some of the plurality of graphics processing cores to:
receive a first workload prepared by a virtual reality application when a rate of movement of a user's head exceeds a threshold level, and receive a second workload prepared by the virtual reality application when the rate of movement does not exceed the threshold level;
render the first workload, comprising to render, from the first workload, a first plurality of frames having a reduced pixel density in a rendering engine, including to pixel shade the first workload at a first shading rate;
provide the first plurality of frames to a display engine;
thereafter send the first plurality of frames to a display panel;
render the second workload, comprising to render, from the second workload, a second plurality of frames in the rendering engine, the rendering engine to render at least some of the second plurality of frames comprising separate surfaces, including to pixel shade the second workload at a second shading rate, the second shading rate twice the first shading rate;
provide the second plurality of frames to the display engine;
combine the separate surfaces of the at least some of the second plurality of frames in the display engine; and
thereafter send the second plurality of frames to the display panel.

9. The graphics processing unit of claim 8 , further comprising scheduler logic to schedule groups of instructions.

10. The graphics processing unit of claim 9 , further comprising a plurality of arithmetic logic units coupled to the plurality of register files, wherein the plurality of arithmetic logic units are to perform operations on integer data types.

11. The graphics processing unit of claim 8 , wherein the non-transitory storage medium further comprises instructions that when executed cause the at least some of the plurality of graphics processing cores to pixel shade the first workload at a lower resolution than the second workload.

12. The graphics processing unit of claim 8 , wherein the non-transitory storage medium further comprises instructions that when executed cause the at least some of the plurality of graphics processing cores to:
render a first region of a first frame of the first plurality of frames at a first resolution; and
render a second region of the first frame at a second resolution lower than the first resolution, the first region comprising a foveated region.

13. The graphics processing unit of claim 12 , wherein the non-transitory storage medium further comprises instructions that when executed cause the at least some of the plurality of graphics processing cores to separately send the first region comprising a foreground surface and the second region comprising a background surface to the display panel, wherein the display panel comprises a head-mounted display having a first buffer to store the foreground surface and a second buffer to store the background surface.

14. The graphics processing unit of claim 8 , wherein the non-transitory storage medium further comprises instructions that when executed cause the at least some of the plurality of graphics processing cores to render the first plurality of frames only in a foveated region.

15. A graphics processor, comprising:
a register file to store information;
a plurality of arithmetic logic units coupled to the register file;
a plurality of texture units coupled to the register file; and
a non-transitory storage medium comprising instructions that when executed cause the graphics processor to:
receive a first workload prepared by a virtual reality application when a rate of movement of a user's head exceeds a threshold level, and receive a second workload prepared by the virtual reality application when the rate of movement

Assignees

• Intel Corp

Inventors

• Babu Ravindra A
• Ms SASHANK
• Musunuri Satyanantha R
• Pawar Sagar C
• Kaipa Kalyan K
• Balakrishnan Vijayakumar
• Kp Sameer

Classifications

• G06T1/20

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G02B27/017

  Head mounted · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/012 Primary

  Head tracking input arrangements · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title