Systems and methods for foveated rendering

Patent metadata

Field	Value
Publication number	US-11862128-B2
Application number	US-202117377288-A
Country	US
Kind code	B2
Filing date	Jul 15, 2021
Priority date	Aug 13, 2019
Publication date	Jan 2, 2024
Grant date	Jan 2, 2024

Abstract

In one embodiment, a computing system may determine a focus point of a viewer based on received sensor data. The system may determine, for a current frame, a first viewing region encompassing a focus point of the viewer and a second view region excluding the first viewing region. The system may determine, for the current frame, color values for the first viewing region using respective first sampling resolutions, and color values for the second viewing region using respective second sampling resolutions. At least one second sampling resolution may be lower than a corresponding first sampling resolution associated with a same color channel. At least two of the second sampling resolutions for the color channels of the second viewing region may be different from each other. The system may output the color values for the first viewing region and the second viewing region of the current frame for display.

First claim

What is claimed is:

1. A method comprising, by a computing system:
determining a focus point of a viewer based on sensor data received by the computing system;
determining, for a current frame, a first viewing region encompassing the focus point of the viewer and a second viewing region excluding the first viewing region;
determining, for the current frame, color values for the first viewing region using respective first sampling resolutions for a plurality of color channels, and color values for the second viewing region using respective second sampling resolutions for the plurality of color channels, wherein at least one second sampling resolution is lower than a corresponding first sampling resolution associated with a same color channel, and wherein at least two of the second sampling resolutions for the plurality of color channels of the second viewing region are different from each other, wherein a first color value associated with a first color channel of the second viewing region is shared in an n×n pixel array in the second viewing region, and wherein the value of n is determined based on a second sampling resolution associated with the first color channel of the n×n pixel array in the second viewing region; and
outputting the color values for the first viewing region and the second viewing region of the current frame for display.

2. The method of claim 1 , wherein the color values corresponding to the first and second viewing regions are determined based on a plurality of tile-surface pairs, and wherein the plurality of tile-surface pairs are determined by:
casting a plurality of rays to a plurality of surfaces for generating the current frame;
determining intersections of the plurality of rays and the plurality of surfaces; and
determining the plurality of tile-surface pairs based on the intersections of the plurality of rays and the plurality of surfaces.

3. The method of claim 2 , wherein the color values corresponding to the second viewing region are determined using a smaller amount of computation than the color values corresponding to the first viewing region.

4. The method of claim 2 , wherein the color values corresponding to the second viewing region are determined based on a smaller number of casted rays than the color values corresponding to the first viewing region.

5. The method of claim 1 , wherein color values of the first color channel of the first viewing region and color values of the first color channel of the second viewing region are determined using a same sampling resolution.

6. The method of claim 1 , wherein color values of a second color channel of the first viewing region and color values of the second color channel of the second viewing region are determined using two sampling resolutions different from each other, and wherein the two sampling resolutions have a relationship of powers of two.

7. The method of claim 1 , further comprising:
determining a color value for each pixel within the n×n pixel array based on a replication process performed by a second computing system on a display.

8. The method of claim 1 , further comprising:
determining a color value for each pixel within the n×n pixel array by interpolating color values associated with the n×n pixel array and one or more adjacent n×n pixel arrays.

9. The method of claim 1 , further comprising:
determining a third viewing region surrounding the second viewing region, wherein color values of the plurality of color channels corresponding to the third viewing region are determined using a same third sampling resolution.

10. The method of claim 1 , further comprising, prior to determining the color values of the first viewing region and the second viewing region:
pre-processing a source data with successively lower sampling resolutions for generating the current frame; and
accessing the source data with the successively lower sampling resolutions while generating the current frame.

11. The method of claim 1 , further comprising:
applying a sharpness filter to a plurality of pixels of the current frame in the second viewing region, wherein the current frame preserves a contrast level on one or more edges associated with one or more objects in the second viewing region.

12. The method of claim 1 , further comprising:
applying a sharpness filter to a plurality of pixels of the current frame in the second viewing region, wherein the current frame preserves an average brightness in the second viewing region.

13. The method of claim 1 , wherein the plurality of color channels comprise a red color channel, a blue color channel, and a green color channel, wherein the red color channel and the blue color channel of the second viewing region have a same sampling resolution, and wherein the green color channel has a higher sampling resolution than the red color channel and the blue color channel.

14. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
determine a focus point of a viewer based on sensor data received by a computing system;
determine, for a current frame, a first viewing region encompassing the focus point of the viewer and a second viewing region excluding the first viewing region;
determine, for the current frame, color values for the first viewing region using respective first sampling resolutions for a plurality of color channels, and color values for the second viewing region using respective second sampling resolutions for the plurality of color channels, wherein at least one second sampling resolution is lower than a corresponding first sampling resolution associated with a same color channel, and wherein at least two of the second sampling resolutions for the plurality of color channels of the second viewing region are different from each other, wherein a first color value associated with a first color channel of the second viewing region is shared in an n×n pixel array in the second viewing region, and wherein the value of n is determined based on a second sampling resolution associated with the first color channel of the n×n pixel array in the second viewing region; and
output the color values for the first viewing region and the second viewing region of the current frame for display.

15. The media of claim 14 , wherein the color values corresponding to the first and second viewing regions are determined based on a plurality of tile-surface pairs, and wherein the plurality of tile-surface pairs are determined by:
casting a plurality of rays to a plurality of surfaces for generating the current frame;
determining intersections of the plurality of rays and the plurality of surfaces; and
determining the plurality of tile-surface pairs based on the intersections of the plurality of rays and the plurality of surfaces.

16. The media of claim 15 , wherein the color values corresponding to the second viewing region are determined using a smaller amount of computation than the color values corresponding to the first viewing region.

17. A computing system comprising:
one or more processors, and
one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:
determine a focus point of a viewer based on sensor data received by the computing system;
determine, for a current frame, a first viewing region encompassing the focus point of the viewer and a second viewing region excluding the first viewing region;
determine, for the current frame, color value

Assignees

• Meta Platforms Tech Llc

Inventors

• Seiler Larry

Classifications

• G09G5/37 Primary

  Details of the operation on graphic patterns (G09G5/38 takes precedence) · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06T1/20

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06T5/003

  Physics · mapped topic

• G06T15/06 Primary

  Ray-tracing · CPC title