Temporal supersampling for foveated rendering systems

Patent metadata

Field	Value
Publication number	US-10713752-B2
Application number	US-201815905801-A
Country	US
Kind code	B2
Filing date	Feb 26, 2018
Priority date	Jun 9, 2017
Publication date	Jul 14, 2020
Grant date	Jul 14, 2020

Abstract

Methods and systems are provided for using temporal supersampling to increase a displayed resolution associated with peripheral region of a foveated rendering view. A method for enabling reconstitution of higher resolution pixels from a low resolution sampling region for fragment data is provided. The method includes an operation for receiving a fragment from a rasterizer of a GPU and for applying temporal supersampling to the fragment with the low resolution sampling region over a plurality of prior frames to obtain a plurality of color values. The method further includes an operation for reconstituting a plurality of high resolution pixels in a buffer that is based on the plurality of color values obtained via the temporal supersampling. Moreover, the method includes an operation for sending the plurality of high resolution pixels for display.

First claim

What is claimed is:

1. A method for drawing higher resolution pixels from a low resolution sampling region, comprising:
processing a fragment of a frame within a graphics pipeline when executing a video game;
applying temporal supersampling to the fragment within the low resolution sampling region over the frame and a plurality of prior frames for obtaining a plurality of color values, wherein the temporal supersampling includes sampling at different locations within the low resolution sampling region for the frame and the plurality of prior frames, wherein each of the frame and the plurality of prior frames includes low resolution pixel data for the low resolution sampling region;
reconstituting a plurality of high resolution pixels associated with the low resolution sampling region in a buffer based on the plurality of color values obtained via the temporal supersampling; and
sending, from the buffer, the plurality of high resolution pixels for presentation on a display.

2. The method of claim 1 , wherein the temporal supersampling includes sampling the different locations for the low resolution sampling region as determined by pixel reprojection.

3. The method of claim 1 , wherein the temporal supersampling includes sampling the different locations for the low resolution sampling region as determined by jitter.

4. The method of claim 1 , wherein the plurality of high resolution pixels is greater in number than the number of the plurality of prior frames.

5. The method of claim 1 , wherein the reconstituting of the plurality of high resolution pixels further includes blending of the plurality of color values.

6. The method of claim 1 , wherein the plurality of high resolution pixels is associated with a native resolution of the display or a resolution that is greater than the native resolution of the display.

7. The method claim 1 , wherein the display is associated with a head mounted display (HMD).

8. The method of claim 1 , wherein the low resolution sampling region is associated with a peripheral region of a foveated rendering view.

9. A graphics system, comprising:
a graphics processing unit (GPU) for processing a fragment of a frame when executing a video game, the GPU applying temporal supersampling to the fragment within a low resolution sampling region over the frame and a plurality of prior frames to obtain a plurality of color values, wherein the temporal supersampling includes sampling at different locations within the low resolution sampling region for the frame and the plurality of prior frames;
a frame buffer for storing the plurality of color values generated for the frame and the plurality of prior frames that are rendered by the GPU; and
a display buffer for storing color values for a plurality of high resolution pixels in the low resolution sampling region that is reconstituted based on the plurality of color values obtained via the temporal supersampling of prior frames,
wherein the plurality of high resolution pixels is configured for presentation of the frame on a display.

10. The graphics system of claim 9 , wherein the temporal supersampling defines the different locations for the low resolution sampling region as determined by pixel reprojection.

11. The graphics system of claim 9 , wherein the temporal supersampling defines the different locations for the low resolution sampling region as determined by jitter.

12. The graphics system of claim 9 , wherein the plurality of high resolution pixels is greater in number that the number of the plurality of prior frames.

13. The graphics system of claim 9 , wherein the plurality of high resolution pixels that is reconstituted includes blending of the plurality of color values.

14. The graphics system of claim 9 , wherein the plurality of high resolution pixels is associated with a native resolution of the display or a resolution that is greater than the native resolution of the display.

15. The graphics system of claim 9 , wherein the display is associated with a head mounted display (HMD).

16. The graphics system of claim 9 , wherein the low resolution sampling region is associated with a peripheral region of a foveated rendering view.

17. A non-transitory computer-readable storage medium storing a computer program executable by a processor-based system, comprising:
program instructions for processing a fragment of a frame within a graphics pipeline when executing a video game, at least a portion of the fragment being located within a low resolution sampling region;
program instructions for applying temporal supersampling to the fragment over the frame and a plurality of prior frames for obtaining a plurality of color values, wherein the temporal supersampling includes sampling at different locations within the low resolution sampling region for the frame and the plurality of prior frames, wherein each of the frame and the plurality of prior frames includes low resolution pixel data for the low resolution sampling region;
program instructions for reconstituting, in a buffer, a plurality of high resolution pixels associated with the low resolution sampling region, the plurality of high resolution pixels based on the plurality of color values obtained via the temporal supersampling; and
program instructions for sending, from the buffer, the plurality of high resolution pixels for presentation on a display.

18. The non-transitory computer-readable medium of claim 17 , wherein the temporal supersampling defines the different locations for the low resolution sampling region over the frame and the plurality of prior frames as determined by pixel reprojection or as determined by jitter, and
wherein the plurality of high resolution pixels is associated with a native resolution of the display or a resolution that is greater than the native resolution of the display.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Young Andrew
• Ho Chris
• Stafford Jeffrey Roger

Classifications

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06T3/4053

  based on super-resolution, i.e. the output image resolution being higher than the sensor resolution · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06T11/40

  Filling planar surfaces by adding surface attributes, e.g. adding colours or textures · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title