Motion based adaptive rendering

Patent metadata

Field	Value
Publication number	US-9928610-B2
Application number	US-201514743700-A
Country	US
Kind code	B2
Filing date	Jun 18, 2015
Priority date	Jun 27, 2014
Publication date	Mar 27, 2018
Grant date	Mar 27, 2018

Abstract

An apparatus, system and method is provided to determine a motion of pixels in local regions of a scene, classify the motion into a speed category, and make decisions on how to render blocks of pixels. In one implementation the motion in a tile is classified into at least three different speed regimes. If the pixels in a tile are in a quasi-static speed regime, a determination is made whether or not to reuse a fraction of pixels from the previous frame. If the pixels are determined to be in a high speed regime, a decision is made whether or not a sampling rate may be reduced.

First claim

What is claimed is:

1. A method of graphics processing utilizing a graphics processing unit (GPU), the method comprising:
determining, by the GPU, on a tile-by-tile basis, a relative speed of objects as measured in screen space co-ordinates or pixels in a current frame relative to a previous frame, the frames corresponding to images to be displayed on a display device;
classifying, by the GPU, each tile within the current frame into one of at least three speed categories based on the determined relative speed, the at least three categories including a quasi-static speed category, a medium speed category, and a high speed category;
making, by the GPU, a sampling decision for each tile, based at least in part on the speed category associated with each tile, including whether the tile is to be sampled at a full resolution sampling rate of at least one sample per pixel in the current frame or sampled at a lower rate in the current frame; and
rendering, by the GPU, tiles of the current frame.

2. The method of claim 1 , wherein for a tile classified in the quasi-static speed category the method further comprises reusing pixel data from the previous frame by mapping a location of at least one pixel in the current frame to a corresponding pixel in the previous frame and copying pixel data for the corresponding pixel of the previous frame to the current frame.

3. The method of claim 2 , wherein a subset of the pixels from a tile of the previous frame are copied to a tile of the current frame based on a pre-selected ratio of rendered to unrendered pixels.

4. The method of claim 2 , further comprising performing a discrepancy check with respect to pixel color values of the previous frame as a condition for reusing pixel data from the previous frame.

5. The method of claim 1 , wherein the speed is determined from a difference in vertex position data for rendered geometry between the current frame and a previous frame.

6. The method of claim 1 , wherein, for a tile classified in the high speed category, making a sampling decision includes selecting a sampling pattern in which a number of rendered samples is less than the number of pixels associated with the tile.

7. The method of claim 1 , further comprising performing edge detection in depth/Z and wherein the making a sampling decision for each tile is further based on whether edges are detected within individual tiles.

8. The method of claim 7 , wherein:
for a tile in the high speed category not including an edge, selecting a sampling pattern in which a number of samples is less than the number of pixels associated with the tile; and
for a tile in the high speed category including an edge, selecting a sampling pattern in which a number of samples is at least equal to one sample per pixel.

9. The method of claim 1 , wherein making a sampling decision further comprises making a decision for each tile whether or not to perform super-sampling.

10. The method of claim 9 , wherein super-sampling is performed for a tile including an edge.

11. A method of graphics processing utilizing a graphics processing unit (GPU), the method comprising:
determining, by the GPU, on a tile-by-tile basis, a speed of objects as measured in screen space pixels in a current frame relative to a previous frame, the frames to be displayed on a display device;
detecting, by the GPU, on a tile-by-tile basis, whether a tile includes an edge;
classifying, by the GPU based on the determined speed, each tile within the current frame into one of at least three speed categories including a quasi-static speed category, an intermediate speed regime, and at least one high speed regime;
making, by the GPU, a sampling rate decision for individual tiles based on the speed category associated with a tile and whether the tile includes an edge, the sampling decision including determining whether the tile is to be sampled at a full resolution sampling rate of at least one sample per pixel in the current frame or sampled at a lower rate in the current frame; and
rendering, by the GPU, tiles of the current frame.

12. The method of claim 11 , wherein for a tile classified in the quasi-static speed category the method further comprises reusing pixel data from the previous frame by mapping a location of at least one pixel in a current frame to a corresponding pixel in the previous frame and copying pixel data for the corresponding pixel of the previous frame to the current frame.

13. The method of claim 12 , wherein a subset of the pixels from a tile of the previous frame are copied to a tile of the current frame based on a pre-selected ratio of rendered to unrendered pixels.

14. The method of claim 12 , further comprising performing a discrepancy check with respect to pixel color values of the previous frame as a condition for reusing pixel data from the previous frame.

15. The method of claim 11 , wherein for a tile classified in the intermediate speed regime, every pixel is sampled at least once.

16. The method of claim 11 , wherein for a tile classified in the at least one high speed regime selecting a sampling pattern in which a number of samples is less than the number of pixels associated with the tile and performing interpolation to determine a color at pixel locations not sampled.

17. The method of claim 11 , further comprising:
for a tile in the at least one high speed regime not including an edge, selecting a sampling pattern in which a number of samples is less than the number of pixels associated with the tile; and
for a tile in the at least one high speed regime including an edge, selecting a sampling pattern in which a number of samples is at least equal to one sample per pixel.

18. A graphics system including a graphics pipeline, the graphics system comprising:
an adaptive sampling generator; and
a pixel shader coupled to the adaptive sampling generator,
wherein the adaptive sampling generator is configured to determine a required sample rate for each tile from among tiles of an image based at least in part on a speed of pixels of objects in each tile, to select a sample pattern based on the required sample rate, and to determine a sample pattern and a sample rate for an individual tile based on a combination of the speed of the pixels of the objects in the individual tile and whether the individual tile includes an edge, and
wherein a reduced sampling rate is selected for each tile not including an edge and having a speed above a threshold speed.

19. The graphics system of claim 18 , further comprising:
a velocity buffer to compare per-vertex coordinates of vertices of objects in a current frame to corresponding per-vertex coordinates in a previous frame, and to determine a per-pixel velocity of pixels in each tile;
an edge detector to detect a presence of edges in each tile; and
a sample generator to determine, for each tile to be shaded, a sample pattern and sampling rate based on a speed associated with pixels in each tile and the presence of edges.

20. The graphics system of claim 18 , further comprising an advection unit and a reconstruction unit, wherein, for a tile having a velocity below a quasi-static speed limit, a sample pattern having a reduced sampling rate is selected and the reconstruction unit fills in missing pixel data by reusing pixel data from a previous frame via advection.

21. The graphics system of claim 20 , wherein the reconstruction unit identifies a location of a pixel in a previous frame by mapping a change in pixel coordinates.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Golas Abhinav
• Ramani Karthik
• Cheng Christopher T
• Brothers John W
• Zhang Liangjun
• Abraham Santosh
• Chow Ki Fung

Classifications

• G09G5/363

  Graphics controllers · CPC title

• G06T7/32

  using correlation-based methods · CPC title

• G06T7/30

  Determination of transform parameters for the alignment of images, i.e. image registration · CPC title

• G06T15/005

  General purpose rendering architectures · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title