Optimized deferred lighting in a foveated rendering system

Patent metadata

Field	Value
Publication number	US-11222444-B2
Application number	US-201916723106-A
Country	US
Kind code	B2
Filing date	Dec 20, 2019
Priority date	Jun 9, 2017
Publication date	Jan 11, 2022
Grant date	Jan 11, 2022

Abstract

A method for implementing a graphics pipeline. The method includes determining a plurality of light sources affecting a virtual scene. Geometries of objects of an image of the scene is projected onto a plurality of pixels of a display from a first point-of-view. The pixels are partitioned into a plurality of tiles. A foveal region of highest resolution is defined for the image as displayed, wherein a first subset of pixels is assigned to the foveal region, and wherein a second subset of pixels is assigned to a peripheral region that is outside of the foveal region. A first set of light sources is determined from the plurality of light sources that affect one or more objects displayed in a first tile that is in the peripheral region. At least two light sources from the first set is clustered into a first aggregated light source affecting the first tile when rendering the image in pixels of the first tile.

First claim

What is claimed is:

1. A method for implementing a graphics pipeline, comprising:
determining a plurality of light sources affecting a virtual scene, wherein each of the plurality of light sources is a virtual light source used when rendering an image of the virtual scene in the graphics pipeline;
projecting geometries of objects of the image of the virtual scene onto a plurality of pixels from a first point-of-view;
determining that a geometry is projected onto a pixel that is in a peripheral region that is outside a foveal region of highest resolution;
clustering a first plurality of light sources affecting the geometry into an aggregated light source when rendering the geometry through the graphics pipeline.

2. The method of claim 1 , wherein the geometry is a primitive of an object.

3. The method of claim 1 , wherein the geometry is a pixel sized fragment of an object.

4. The method of claim 1 , further comprising:
determining that a second geometry is projected onto a second pixel that is in the foveal region from the first point-of-view; and
rendering the second geometry through the graphics pipeline using a second plurality of light sources affecting the second geometry.

5. The method of claim 1 , further comprising:
generating a plurality of clusters of light sources from the plurality of light sources using a nearest neighbor search or a k-means clustering technique, wherein each cluster of light sources is associated with a corresponding aggregated light source, wherein the each cluster of light sources comprises at least one light source.

6. The method of claim 1 , wherein the foveal region in the image corresponds to where on a display an eye of a user is directed.

7. The method of claim 1 , wherein the determining that the geometry is projected onto the pixel that is in the peripheral region includes:
partitioning the plurality of pixels of a display into a plurality of tiles; and
determining that the plurality of light sources affect one or more objects displayed in a first tile that is in the peripheral region, wherein the first tile includes the pixel.

8. A non-transitory computer-readable medium storing a computer program for implementing a graphics pipeline, the computer-readable medium comprising:
program instructions for determining a plurality of light sources affecting a virtual scene, wherein each of the plurality of light sources is a virtual light source used when rendering an image of the virtual scene in the graphics pipeline;
program instructions for projecting geometries of objects of the image of the virtual scene onto a plurality of pixels from a first point-of-view;
program instructions for determining that a geometry is projected onto a pixel that is in a peripheral region that is outside a foveal region of highest resolution;
program instructions for clustering a first plurality of light sources affecting the geometry into an aggregated light source when rendering the geometry through the graphics pipeline.

9. The non-transitory computer-readable medium of claim 8 , wherein in the program instructions the geometry is a primitive of an object.

10. The non-transitory computer-readable medium of claim 8 , wherein in the program instructions the geometry is a pixel sized fragment of an object.

11. The non-transitory computer-readable medium of claim 8 , further comprising:
program instructions for determining that a second geometry is projected onto a second pixel that is in the foveal region from the first point-of-view; and
program instructions for rendering the second geometry through the graphics pipeline using a second plurality of light sources affecting the second geometry.

12. The non-transitory computer-readable medium of claim 8 , further comprising:
program instructions for generating a plurality of clusters of light sources from the plurality of light sources using a nearest neighbor search or a k-means clustering technique, wherein each cluster of light sources is associated with a corresponding aggregated light source, wherein the each cluster of light sources comprises at least one light source.

13. The non-transitory computer-readable medium of claim 8 , wherein in the program instructions the foveal region in the image corresponds to where on a display an eye of a user is directed.

14. The non-transitory computer-readable medium of claim 8 , wherein the program instructions for determining that the geometry is projected onto the pixel that is in the peripheral region includes:
program instructions for partitioning the plurality of pixels of a display into a plurality of tiles; and
program instructions for determining that the plurality of light sources affect one or more objects displayed in a first tile that is in the peripheral region, wherein the first tile includes the pixel.

15. A computer system comprising:
a processor; and
memory coupled to the processor and having stored therein instructions that, if executed by the computer system, cause the computer system to execute a method for implementing a graphics pipeline, comprising:
determining a plurality of light sources affecting a virtual scene, wherein each of the plurality of light sources is a virtual light source used when rendering an image of the virtual scene in the graphics pipeline;
projecting geometries of objects of the image of the virtual scene onto a plurality of pixels from a first point-of-view;
determining that a geometry is projected onto a pixel that is in a peripheral region that is outside a foveal region of highest resolution;
clustering a first plurality of light sources affecting the geometry into an aggregated light source when rendering the geometry through the graphics pipeline.

16. The computer system of claim 15 , wherein in the method the geometry is a primitive of an object or a pixel sized fragment of the object.

17. The computer system of claim 15 , the method further comprising:
determining that a second geometry is projected onto a second pixel that is in the foveal region from the first point-of-view; and
rendering the second geometry through the graphics pipeline using a second plurality of light sources affecting the second geometry.

18. The computer system of claim 15 , the method further comprising:
generating a plurality of clusters of light sources from the plurality of light sources using a nearest neighbor search or a k-means clustering technique, wherein each cluster of light sources is associated with a corresponding aggregated light source, wherein the each cluster of light sources comprises at least one light source.

19. The computer system of claim 15 , wherein in the method the foveal region in the image corresponds to where on a display an eye of a user is directed.

20. The computer system of claim 15 , wherein in the method the determining that the geometry is projected onto the pixel that is in the peripheral region includes:
partitioning the plurality of pixels of a display into a plurality of tiles; and
determining that the plurality of light sources affect one or more objects displayed in a first tile that is in the peripheral region, wherein the first tile includes the pixel.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Young Andrew
• Ho Chris
• Stafford Jeffrey Roger

Classifications

• G06T11/10 Primary

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06T13/80 Primary

  Two-dimensional [2D] animation, e.g. using sprites · CPC title

• G06T2215/12

  Shadow map, environment map · CPC title

• G06T15/50

  Lighting effects · CPC title

• G06T1/20

  Processor architectures; Processor configuration, e.g. pipelining · CPC title