Foveated rendering in tiled architectures

Patent metadata

Field	Value
Publication number	US-11222397-B2
Application number	US-201615390333-A
Country	US
Kind code	B2
Filing date	Dec 23, 2016
Priority date	Dec 23, 2016
Publication date	Jan 11, 2022
Grant date	Jan 11, 2022

Abstract

This disclosure describes examples for foveation rendering. A graphics processing unit (GPU) may render image content for each tile at different sizes as part of the rendering pass after a binning pass in which the GPU determined to which tiles vertices of primitives belong. The GPU may upsample the rendered image content based on the size at which GPU rendered the image content, and output the upsampled image content for later display.

First claim

What is claimed is:

1. A method of foveation rendering, the method comprising:
during a binning pass, dividing an image surface into a plurality of tiles, wherein image content of the image surface is to be rendered during a rendering pass;
determining that a first tile of the plurality of tiles of the image surface is to include image content having a first resolution, wherein the first resolution is less than a second resolution for other tiles of the plurality of tiles of the image surface;
determining a size at which to render the image content for the first tile, wherein the size at which to render the image content for the first tile is based on the first resolution of the image content in the first tile, wherein the size at which to render the image content is smaller than a size of the first tile, and wherein the image content for the first tile is part of the image content of the image surface, which was divided during the binning pass, and the image content is to be rendered during the rendering pass;
during the rendering pass, rendering the image content for the first tile at the determined size that is smaller than the size of the first tile and based on the first resolution of the image content in the first tile, wherein the image content rendered for the entire first tile covers less than the entire first tile, wherein the image content rendered for the entire first tile does not overlap image content for any other tile, wherein rendering the image content comprises rendering the image content for the first tile to a portion of memory, wherein the portion of the memory is less than the entire memory for the first tile, and wherein the size of the rendered image content is less than the size of the entire first tile;
upsampling the image content in the first tile to generate upsampled image content;
interpolating the upsampled image content based on the upsampled image content in the first tile to generate interpolated image content; and
outputting the interpolated image content.

2. The method of claim 1 , further comprising:
defining a viewport for the first tile based on the first resolution of the image content in the first tile and the size of the first tile,
wherein rendering the image content for the first tile at the size that is smaller than the size of the first tile comprises rendering the image content based on the defined viewport for the first tile.

3. The method of claim 2 , wherein defining the viewport comprises:
multiplying a ratio between the first resolution of the image content in the first tile and a full resolution of the image content by the size of the first tile to determine a size of the viewport.

4. The method of claim 1 , further comprising:
determining resolutions for different portions of the image surface; and
during the binning pass, and prior to the rendering, determining which vertices of primitives of the image surface belong to the first tile,
wherein determining that the first tile of the image surface is to include image content having the first resolution comprises determining that the first tile of the image surface is to include image content having the first resolution based on the determination of resolutions for different portions of the image surface and the determination of which vertices of the image surface belong to the first tile.

5. The method of claim 4 , further comprising:
receiving information indicative of the resolutions for portions of the image surface,
wherein determining resolutions for different portions of the image surface comprises determining resolutions for different portions of the image surface based on the received information.

6. The method of claim 1 , wherein upsampling the image content comprises upsampling as part of the outputting of the upsampled image content.

7. The method of claim 1 , wherein the upsampled image content comprises a first upsampled image content, the method further comprising:
determining that a second tile of the image surface is to include image content having the second resolution;
rendering the image content for the second tile at a size that is smaller than the size of the second tile and based on the second resolution of the image content in the second tile;
upsampling the image content in the second tile to generate a second upsampled image content; and
outputting the second upsampled image content.

8. The method of claim 7 , wherein upsampling the image content in the first tile and outputting the first upsampled image content comprises upsampling the image content in the first tile and outputting the first upsampled image content after rendering the image content for the second tile.

9. The method of claim 1 , further comprising displaying the upsampled image content.

10. A device for foveation rendering, the device comprising:
a system memory; and
a graphics processing unit (GPU) comprising at least one of programmable or fixed-function circuitry, wherein the GPU is configured to:
during a binning pass, divide an image surface into a plurality of tiles, wherein image content of the image surface is to be rendered during a rendering pass;
determine that a first tile of the plurality of tiles of the image surface is to include image content having a first resolution, wherein the first resolution is less than a second resolution for other tiles of the plurality of tiles of the image surface;
determine a size at which to render the image content for the first tile, wherein the size at which to render the image content for the first tile is based on the first resolution of the image content in the first tile, wherein the size at which to render the image content is smaller than a size of the first tile, and wherein the image content for the first tile is part of the image content of the image surface, which was divided during the binning pass, and the image content is to be rendered during the rendering pass;
during the rendering pass, render the image content for the first tile at the determined size that is smaller than the size of the first tile and based on the first resolution of the image content in the first tile, wherein the image content rendered for the entire first tile covers less than the entire first tile, wherein the image content rendered for the entire first tile does not overlap image content for any other tile, wherein to render the image content, the GPU is configured to render the image content for the first tile to a portion of memory for the GPU, wherein the portion of the memory for the GPU is less than the entire memory for the first tile in the system memory, and wherein the size of the rendered image content is less than the size of the entire first tile;
upsample the image content in the first tile to generate upsampled image content;
interpolate the upsampled image content based on the upsampled image content in the first tile to generate interpolated image content; and
output the interpolated image content for storage in the system memory.

11. The device of claim 10 , further comprising a processor configured to define a viewport for the first tile based on the first resolution of the image content in the first tile and the size of the first tile,
wherein to render the image content for the first tile at the size that is smaller than the size of the first tile, the GPU is configured to render the image content based on the defined viewport for the first tile.

12. The device of claim 11 , wherein to define the viewport, the processor is configured to multiply a ratio between the first resolution of the image content in the first tile and a full resolution of the image content by the size of the first ti

Assignees

• Qualcomm Inc

Inventors

• Saleh Skyler Jonathon
• Garcia Garcia David Rigel
• Ribble Maurice Franklin

Classifications

• G06T11/40 Primary

  Filling planar surfaces by adding surface attributes, e.g. adding colours or textures · CPC title

• G06T15/20 Primary

  Perspective computation · CPC title

• H04N1/3876

  Recombination of partial images to recreate the original image · CPC title

• H04N1/3935

  with modification of image resolution, i.e. determining the values of picture elements at new relative positions · CPC title

• G06T15/005 Primary

  General purpose rendering architectures · CPC title