Depth based foveated rendering for display systems

Patent metadata

Field	Value
Publication number	US-12547005-B2
Application number	US-202418959036-A
Country	US
Kind code	B2
Filing date	Nov 25, 2024
Priority date	Mar 22, 2017
Publication date	Feb 10, 2026
Grant date	Feb 10, 2026

Abstract

Methods and systems for depth-based foveated rendering in a display system are disclosed. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes using different wavefront divergence. Some embodiments include monitoring eye orientations of a user of the display system. A fixation point can be determined based on the eye orientations, the fixation point representing a three-dimensional location with respect to a field of view. Location information of virtual object(s) to present is obtained, with the location information including three-dimensional position(s) of the virtual object(s). A resolution of the virtual object(s) can be adjusted based on a proximity of the location(s) of the virtual object(s) to the fixation point. The virtual object(s) are presented by the display system according to the adjusted resolution(s).

First claim

What is claimed is:

1 . A method for presenting virtual content through a display device, the method performed by one or more processors and comprising:
determining an amount of light reaching a retina of an eye of a user of the display device; adjusting resolution of virtual content to be presented to the user based on the amount of light reaching the retina, wherein adjusting the resolution of the virtual content includes increasing a spatial resolution of the virtual content with increasing amounts of light reaching the retina; and causing presentation, via the display device, of the virtual content to the user, wherein the virtual content is presented according to the adjusted resolution.

2 . The method of claim 1 , wherein the display device includes:
a light modulating system configured to output light to form images; and one or more waveguides configured to receive the light from the light modulating system and to output the light across a surface of the one or more waveguides.

3 . The method of claim 2 , wherein the one or more waveguides comprise a stack of waveguides, wherein at least some of the waveguides of the stack of waveguides provide different amounts of wavefront divergence than other waveguides of the stack of waveguides.

4 . The method of claim 2 , wherein determining the amount of light reaching the retina further comprises determining an amount of light that is output by the one or more waveguides to the eye of the user.

5 . The method of claim 1 , wherein determining the amount of light reaching the retina comprises measuring the ambient illumination level using an outwardly-directed camera of the display device.

6 . The method of claim 1 , wherein determining the amount of light reaching the retina comprises determining a size of a pupil of the eye of the user.

7 . The method of claim 1 , further comprising:
determining a user fixation point; and obtaining location information of the virtual content, the location information indicating three-dimensional positions of the virtual content, wherein adjusting resolution of the virtual content further comprises varying resolutions of the virtual content based on a proximity of the virtual content to the user fixation point.

8 . The method of claim 1 , wherein adjusting resolution of the virtual content further comprises decreasing resolution of the virtual content with increasing distance of the virtual content from the user fixation point.

9 . The method of claim 8 , wherein a trend of the decreasing resolution is substantially similar to a trend of decreasing cone density in the retina of the user.

10 . The method of claim 9 , wherein a zone of maximum resolution within a user field of view is centered on a point corresponding to a foveola of the eye of the user, wherein the resolution of virtual content within a user field of view decreases for virtual content disposed outside of ±20° of the point.

11 . The method of claim 8 , wherein decreasing resolution comprises decreasing polygon count at all levels of light reaching the retina.

12 . The method of claim 1 , wherein adjusting resolution further comprises setting the resolution at one of at least two resolution levels.

13 . The method of claim 1 , wherein adjusting resolution of the virtual content further comprises decreasing one or both of a color depth and a contrast ratio with decreasing amounts of light reaching the retina.

14 . The method of claim 1 , further comprising:
determining whether the amount of light corresponds to photopic, mesotopic, or scotopic illumination levels, wherein adjusting resolution of the virtual content further comprises setting the resolutions based upon whether the illumination level corresponds to a photopic, mesotopic, or scotopic illumination level.

15 . The method of claim 1 , wherein adjusting resolution of the virtual content further comprises, at a scotopic illumination level, displaying the virtual content using a single color.

16 . The method of claim 1 , wherein adjusting resolution of the virtual content further comprises decreasing a contrast ratio of images forming the virtual content as the amount of light reaching the retina of the eye decreases.

17 . The method of claim 1 , further comprising:
providing virtual content in a plurality of component colors, wherein adjusting resolution of the virtual content further comprises providing different resolutions for different component colors.

18 . The method of claim 1 , further comprising:
preventing rendering of virtual content located within an opposite peripheral region for each eye of the user, wherein the opposite peripheral region is on a side of the user opposite a side of the user on which the eye is disposed.

19 . The method of claim 1 , further comprising:
preventing rendering of virtual content corresponding to an optic nerve blind spot of the eye.

20 . Non-transitory computer-readable storage media storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations for presenting virtual content through a display device, the operations comprising:
determining an amount of light reaching a retina of an eye of a user of the display device; adjusting resolution of virtual content to be presented to the user based on the amount of light reaching the retina, wherein adjusting the resolution of the virtual content includes increasing a spatial resolution of the virtual content with increasing amounts of light reaching the retina; and causing presentation, via the display device, of the virtual content to the user, wherein the virtual content is presented according to the adjusted resolution.

Assignees

• Magic Leap Inc

Inventors

• Yeoh Ivan Li Chuen
• Edwin Lionel Ernest
• Samec Nicole Elizabeth
• Robaina Nastasja U
• Mathur Vaibhav
• Dalrymple Timothy Mark
• Schaefer Jason
• Carlisle Clinton
• Cheng Hui-Chuan
• Oh Chulwoo
• Premysler Philip
• Zhang Xiaoyang
• Carlson Adam C

Classifications

• G02B2027/0134

  of stereoscopic type · CPC title

• G02B2027/0185

  Displaying image at variable distance · CPC title

• H04N2213/002

  Eyestrain reduction by processing stereoscopic signals or controlling stereoscopic devices · CPC title

• G06T15/00

  Three-dimensional [3D] image rendering · CPC title

• H04N13/398

  Synchronisation thereof; Control thereof · CPC title