Virtual reality apparatus and method including prioritized pixel shader operations, alternate eye rendering, and/or augmented timewarp

Patent metadata

Field	Value
Publication number	US-11880934-B2
Application number	US-202117517113-A
Country	US
Kind code	B2
Filing date	Nov 2, 2021
Priority date	Apr 7, 2017
Publication date	Jan 23, 2024
Grant date	Jan 23, 2024

Abstract

An apparatus and method are described for performing an early depth test on graphics data. For example, one embodiment of a graphics processing apparatus comprises: early depth test circuitry to perform an early depth test on blocks of pixels to determine whether all pixels in the block of pixels can be resolved by the early depth test; a plurality of execution circuits to execute pixel shading operations on the blocks of pixels; and a scheduler circuit to schedule the blocks of pixels for the pixel shading operations, the scheduler circuit to prioritize the blocks of pixels in accordance with the determination as to whether all pixels in the block of pixels can be resolved by the early depth test.

First claim

What is claimed is:

1. A graphics processing apparatus comprising:
an image rendering circuit to render left and right image frames to be viewed by a user's left and right eyes, respectively;
one or more display buffers to store image frames rendered by the image rendering circuit; and
a time warping circuit to perform time warping operations on image frames stored in the one or more display buffers, a time warping operation to transform an image frame in accordance with current sensor data provided from a user/eye tracking module, the time warping circuit to alternate between time warping a left image frame and a right image frame to be viewed by the user's left and right eyes, respectively;
wherein the alternation comprises: as the time warping circuit producing a time warped image frame to a first frame buffer to be displayed for the user's right eye, a non-warped rendered image is displayed from a second frame buffer for the user's left eye, and as the time warping circuit producing a time warped image frame to the second frame buffer to be displayed for the user's left eye, a non-warped rendered image is to be displayed from the first frame buffer for the user's right eye.

2. The graphics processing apparatus as in claim 1 further comprising:
a depth buffer to store depth data associated with one or more objects in the image frames, wherein the time warping circuit is to perform the time warping operations in accordance with the depth data.

3. The graphics processing apparatus as in claim 2 wherein the current sensor data comprises coordinate data indicating a current orientation of the user's head and/or a current gaze of the user's eyes.

4. The graphics processing apparatus as in claim 3 further comprising:
a head mounted display comprising a left display to display the left image frames and a right display to display the right image frames, the user/eye tracking module mounted on the head mounted display.

5. The graphics processing apparatus as in claim 1 wherein the time warping circuit is to alternate between time warping the left image frame and the-right image frames at a specified frame rate.

6. The graphics processing apparatus as in claim 1 wherein the image rendering circuit comprises a rasterization pipeline including a vertex shader, a geometry shader and a rasterizer.

7. The graphics processing apparatus as in claim 1 wherein the image rendering circuit comprises a ray tracing-based pipeline including ray generation circuitry, ray traversal circuitry, and ray intersection circuitry.

8. The graphics processing apparatus as in claim 1 wherein the one or more display buffers comprise a first front buffer to store left image frames and a second front buffer to store right image frames, the time warping circuit to alternate between warping image frames from the first front buffer and the second front buffer.

9. A method comprising:
rendering left and right image frames to be viewed by a user's left and right eyes, respectively;
storing the image frames in one or more display buffers;
alternating, by a time warping circuit, between time warping a left image frame and a right image frame to be viewed by the user's left and right eyes, respectively, the time warping to transform the left and right image frames in accordance with current sensor data provided from a user/eye tracking module; and
wherein the alternating comprises: as the time warping circuit producing a time warped image frame to a first frame buffer to be displayed for the user's right eye, a non-warped rendered image is displayed from a second frame buffer for the user's left eye and wherein as the time warping circuit producing a time warped image frame to the second frame buffer to be displayed for the user's left eye, a non-warped rendered image from the first frame buffer is displayed for the user's right eye.

10. The method as in claim 9 further comprising:
storing depth data associated with one or more objects in the image frames; and
performing the time warping operations in accordance with the depth data.

11. The method as in claim 10 wherein the current sensor data comprises coordinate data indicating a current orientation of the user's head and/or a current gaze of the user's eyes.

12. The method as in claim 11 further comprising:
displaying the left and right image frames on a left display and a right display, respectively, of a head mounted display, the user/eye tracking module mounted on the head mounted display.

13. The method as in claim 9 further comprising:
alternating between time warping the left image frame and the right image frame at a specified frame rate.

14. The method as in claim 9 wherein the operation of rendering left and right image frames includes performing vertex shading and/or geometry shading for primitives and rasterizing the primitives.

15. The method as in claim 9 wherein the operation of rendering left and right image frames includes generating rays, traversing the rays through a scene with primitives, and determining intersections between the rays and primitives.

16. The method as in claim 9 wherein the one or more display buffers comprise a first front buffer to store left image frames and a second front buffer to store right image frames.

17. An apparatus comprising:
a processor and computer-readable medium having instructions stored thereon which, when executed by the processor, causes the processor to perform:
rendering left and right image frames to be viewed by a user's left and right eyes, respectively;
storing the image frames in one or more display buffers;
alternating between time warping a left image frame and a right image frame to be viewed by the user's left and right eyes, respectively, the time warping to transform the left and right image frames in accordance with current sensor data provided from a user/eye tracking module; and
wherein the alternating comprises: as the a time warped image frame is being produced to a first frame buffer to be displayed for the user's right eye, a non-warped rendered image is displayed from a second frame buffer for the user's left eye, and as a time warped image frame is being produced to the second frame buffer to be displayed for the user's left eye, a non-warped rendered image is displayed from the first frame buffer for the user's right eye.

18. The apparatus as in claim 17 wherein the processor is caused to further perform:
storing depth data associated with one or more objects in the image frames; and
performing the time warping operations in accordance with the depth data.

19. The apparatus as in claim 18 wherein the current sensor data comprises coordinate data indicating a current orientation of the user's head and/or a current gaze of the user's eyes.

20. The apparatus as in claim 19 wherein the processor is caused to further perform:
displaying the left and right image frames on a left display and a right display, respectively, of a head mounted display, the user/eye tracking module mounted on the head mounted display.

21. The apparatus as in claim 17 wherein the processor is caused to further perform:
alternating between time warping the left image frame and the right image frame at a specified frame rate.

22. The apparatus as in claim 17 wherein the operation of rendering left and right image frames includes performing vertex shading and/or geometry shading for primitives and rasterizing the primitives.

23. The apparatus as in cl

Assignees

• Intel Corp

Inventors

• Insko Brent E
• Surti Prasoonkumar

Classifications

• G06T15/405 Primary

  using Z-buffer · CPC title

• G06T15/005

  General purpose rendering architectures · CPC title

• H04N13/279

  the virtual viewpoint locations being selected by the viewers or determined by tracking · CPC title

• H04N13/344

  with head-mounted left-right displays · CPC title

• H04N13/383

  for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes · CPC title