Virtual reality apparatus and method including prioritized pixel shader operations, alternate eye rendering, and/or augmented timewarp

Patent metadata

Field	Value
Publication number	US-12437469-B2
Application number	US-202418413252-A
Country	US
Kind code	B2
Filing date	Jan 16, 2024
Priority date	Apr 7, 2017
Publication date	Oct 7, 2025
Grant date	Oct 7, 2025

Abstract

An apparatus and method are described for performing an early depth test on graphics data. For example, one embodiment of a graphics processing apparatus comprises: early depth test circuitry to perform an early depth test on blocks of pixels to determine whether all pixels in the block of pixels can be resolved by the early depth test; a plurality of execution circuits to execute pixel shading operations on the blocks of pixels; and a scheduler circuit to schedule the blocks of pixels for the pixel shading operations, the scheduler circuit to prioritize the blocks of pixels in accordance with the determination as to whether all pixels in the block of pixels can be resolved by the early depth test.

First claim

What is claimed is:

1. A graphics processing apparatus comprising:
an image rendering circuit to render left and right image frames to be viewed by a user's left and right eyes, respectively, in response to a frame completion marker being met; and
a circuit to transform image frames stored in one or more display buffers based on most recent sensor data to track user movement, the circuit to alternate between transforming image frames for the user's left and right eyes based on depth information of pixels and the user movement, in response to the frame completion marker being missed.

2. The graphics processing apparatus of claim 1 , wherein the depth information of the pixels is stored in a depth buffer and the depth information of pixels corresponds to one or more objects in the image frames.

3. The graphics processing apparatus of claim 1 , wherein the image rendering circuit and the circuit to transform the images frames are integrated within a head mounted display.

4. The graphics processing apparatus of claim 1 , wherein alternating between transforming image frames for the user's left and right eyes comprises producing transformed image frames from the circuit to be displayed for one of the user's left and right eyes at a frame rate, and producing a further transformed image frame from the circuit to be displayed for the other of the user's left and right eyes at a further frame rate, the frame rate being different from the further frame rate.

5. The graphics processing apparatus of claim 1 , wherein the image rendering circuit comprises a rasterization pipeline including a vertex shader, a geometry shader and a rasterizer.

6. The graphics processing apparatus of claim 1 , wherein the image rendering circuit comprises a ray tracing-based pipeline including ray generation circuitry, ray traversal circuitry, and ray intersection circuitry.

7. The graphics processing apparatus of claim 1 , wherein alternating between transforming image frames for the user's left and right eyes comprises producing transformed image frames from the circuit to be displayed for one of the user's left and right eyes in response to a color or image transition between frames for the one of the user's left and right eyes, and producing a further transformed image frame from the circuit to be displayed for the other of the user's left and right eyes in response to a further color or image transition between further frames for the other one of the user's left and right eyes.

8. A method comprising:
displaying rendered left and right image frames to be viewed by a user's left and right eyes in response to a time to render a next image frame being less than a threshold time, the threshold time corresponding to a further time to transform a stored image frame, respectively; and
alternating transforming image frames for the user's left and right eyes stored in one or more display buffers based on most recent sensor data to track user movement and depth information of pixels, in response to the time to render the next image frame being greater than the threshold time.

9. The method of claim 8 , wherein the depth information of the pixels is stored in a depth buffer and the depth information of pixels corresponds to one or more objects in the rendered left and right image frames.

10. The method of claim 8 , wherein alternating transforming the image frames is to alternate transforming the image frames for the user's left and right eyes at a specific frame rate.

11. The method of claim 8 , wherein the rendered left and right image frames are rendered by an image rendering circuit that comprises a rasterization pipeline including a vertex shader, a geometry shader and a rasterizer.

12. The method of claim 8 , wherein the rendered left and right image frames are rendered by an image rendering circuit that comprises a ray tracing-based pipeline including ray generation circuitry, ray traversal circuitry, and ray intersection circuitry.

13. The method of claim 8 , wherein the threshold time is a constant time to perform time warp.

14. One or more non-transitory computer-readable medium having instructions stored thereon which, when executed by a processor, causes the processor to perform:
displaying rendered left and right image frames to be viewed by a user's left and right eyes, respectively, in response to a frame completion marker being met; and
alternating transforming image frames stored in one or more display buffers for the user's left and right eyes based on most recent sensor data to track user movement and depth information of pixels, in response to the frame completion marker being missed.

15. The one or more non-transitory computer-readable medium of claim 14 , wherein the depth information of the pixels is stored in a depth buffer and the depth information of pixels corresponds to one or more objects in the rendered left and right image frames.

16. The one or more non-transitory computer-readable medium of claim 14 , wherein the alternating transforming is performed by circuits integrated within a head mounted display.

17. The one or more non-transitory computer-readable medium of claim 14 , wherein the rendered left and right image frames are rendered by an image rendering circuit that comprises a rasterization pipeline including a vertex shader, a geometry shader and a rasterizer.

18. The one or more non-transitory computer-readable medium of claim 14 , wherein the rendered left and right image frames are rendered by an image rendering circuit that comprises a ray tracing-based pipeline including ray generation circuitry, ray traversal circuitry, and ray intersection circuitry.

19. The one or more non-transitory computer-readable medium of claim 14 , wherein the frame completion marker comprises a user-specified application marker to use a rendered image frame for display.

20. The one or more non-transitory computer-readable medium of claim 14 , wherein the frame completion marker specifies a threshold determined based on one or more rendered image frames.

Assignees

• Intel Corp

Inventors

• Insko Brent E
• Surti Prasoonkumar

Classifications

• H04N13/279

  the virtual viewpoint locations being selected by the viewers or determined by tracking · CPC title

• H04N13/398

  Synchronisation thereof; Control thereof · CPC title

• H04N13/383

  for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes · CPC title

• H04N13/344

  with head-mounted left-right displays · CPC title

• G06T15/005

  General purpose rendering architectures · CPC title