Foveated image rendering

Patent metadata

Field	Value
Publication number	US-9727991-B2
Application number	US-201313781763-A
Country	US
Kind code	B2
Filing date	Mar 1, 2013
Priority date	Mar 1, 2013
Publication date	Aug 8, 2017
Grant date	Aug 8, 2017

Abstract

A method and system for foveated image rendering are provided herein. The method includes tracking a gaze point of a user on a display device and generating a specified number of eccentricity layers based on the gaze point of the user. The method also includes antialiasing the eccentricity layers to remove artifacts, rendering a foveated image based on the eccentricity layers, and displaying the foveated image to the user via the display device.

First claim

What is claimed is:

1. A method for foveated image rendering, comprising:
tracking a gaze point of a user on a display device;
generating a specified number of eccentricity layers based on the gaze point of the user, wherein generating the specified number of eccentricity layers comprises alternately updating middle eccentricity layers and outer eccentricity layers;
antialiasing the eccentricity layers to remove artifacts, wherein antialiasing comprises blending pixel values for each eccentricity layer with a temporally reprojected value from a previous frame;
rendering a foveated image based on the eccentricity layers; and
displaying the foveated image to the user via the display device.

2. The method of claim 1 , comprising calculating an angular size and a resolution for each of the eccentricity layers via a visual acuity model.

3. The method of claim 2 , wherein an inner eccentricity layer is smallest in angular size and is rendered at a resolution of the display device, and wherein an outer eccentricity layer is largest in angular size and is rendered at a lowest resolution.

4. The method of claim 1 , wherein generating the specified number of eccentricity layers comprises generating an outermost eccentricity layer that subsamples the entire display device and preserves an aspect ratio of the display device.

5. The method of claim 1 , comprising updating the foveated image as the gaze point of the user changes.

6. The method of claim 1 , wherein generating the specified number of eccentricity layers comprises performing ray tracing with a continuous falloff of acuity to generate a plurality of eccentricity layers.

7. A foveated image rendering system, comprising:
a display device;
an eye tracking device;
a processor that is adapted to execute stored instructions; and
a system memory, wherein the system memory comprises code configured to:
track a gaze point of a user on the display device via the eye tracker;
generate a specified number of eccentricity layers based on the gaze point of the user, wherein generating the specified number of eccentricity layers comprises alternately updating middle eccentricity layers and outer eccentricity layers;

antialias the eccentricity layers to remove artifacts by blending pixel values for each eccentricity layer with a temporally reprojected value from a previous frame;
send the gaze point of the user and the eccentricity layers to the display device; and
wherein the display device is configured to:
render a foveated image based on the eccentricity layers; and
display the foveated image to the user via the display device.

8. The foveated image rendering system of claim 7 , wherein the processor comprises a specialized graphics processing unit (GPU).

9. The foveated image rendering system of claim 7 , wherein an outermost eccentricity layer subsamples the entire display device and preserves an aspect ratio of the display device.

10. The foveated image rendering system of claim 7 , wherein the system memory comprises code configured to:
continuously track the gaze point of the user on the display device via the eye tracker; and
update the foveated image as the gaze point of the user changes.

11. The foveated image rendering system of claim 7 , wherein the system memory comprises code configured to calculate an angular size and a resolution for each eccentricity layer via a visual acuity model.

12. The foveated image rendering system of claim 7 , wherein the display device is configured to update an innermost eccentricity layer at a rate of 120 hertz.

13. The foveated image rendering system of claim 7 , wherein the eye tracker is configured to update the gaze point of the user on the display device at a rate of 300 hertz.

14. The foveated image rendering system of claim 7 , wherein the system memory comprises code configured to render the foveated image and display the foveated image to the user via the display device with a latency, based on the plurality of latency characteristics, that is lower than a specified upper threshold.

15. The foveated image rendering system of claim 7 , wherein the system memory comprises code configured to generate the specified number of eccentricity layers by performing ray tracing with a continuous falloff of acuity to generate a plurality of eccentricity layers.

16. The foveated image rendering system of claim 7 , wherein the system memory comprises code configured to render the foveated image by smoothly blending the eccentricity layers and interpolating the eccentricity layers to a native resolution of the display device.

17. The foveated image rendering system of claim 7 , wherein the system memory comprises code configured to antialias the eccentricity layers by computing geometric coverage and depth information at a higher resolution than an eccentricity layer resolution.

Assignees

• Microsoft Corp
• Microsoft Technology Licensing Llc

Inventors

• Guenter Brian K
• Finch Mark
• Snyder John
• Drucker Steven
• Tan Desney S

Classifications

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06T11/40 Primary

  Filling planar surfaces by adding surface attributes, e.g. adding colours or textures · CPC title