Depth plane selection for multi-depth plane display systems by user categorization

Patent metadata

Field	Value
Publication number	US-12008723-B2
Application number	US-202217962289-A
Country	US
Kind code	B2
Filing date	Oct 7, 2022
Priority date	Aug 3, 2018
Publication date	Jun 11, 2024
Grant date	Jun 11, 2024

Abstract

A display system includes a head-mounted display configured to project light, having different amounts of wavefront divergence, to an eye of a user to display virtual image content appearing to be disposed at different depth planes. The wavefront divergence may be changed in discrete steps, with the change in steps being triggered based upon whether the user is fixating on a particular depth plane. The display system may be calibrated for switching depth planes for a main user. Upon determining that a guest user is utilizing the system, rather than undergoing a full calibration, the display system may be configured to switch depth planes based on a rough determination of the virtual content that the user is looking at. The virtual content has an associated depth plane and the display system may be configured to switch to the depth plane of that virtual content.

First claim

What is claimed is:

1. An augmented reality display system configured to present virtual image content on a plurality of depth planes, the augmented reality display system comprising:
a waveguide configured to present the virtual image content by outputting light to a wearer, the waveguide further configured to pass light from the world into an eye of the wearer;
an imaging device configured to capture images of eyes of the wearer; and
at least one processor configured to:
determine whether the wearer is a calibrated user or a guest user based at least in part on images of the eyes of the wearer from the imaging device;
based on determining that the wearer is a calibrated user:
load pre-existing user depth plane switching calibration information;
identify a fixation point of the user based upon the pre-existing user depth plane switching calibration information; and
switch the virtual image content to be presented at a depth plane that corresponds to the fixation point; and

based on determining that the wearer is a guest user:
determine the interpupillary distance of the guest user;
calculate an estimated fixation point of the eyes of the guest user based upon the determined interpupillary distance, wherein the estimated fixation point is a point in three-dimensional space on which the eyes of the guest user are focused;
determine a plurality of system-defined volumes such that a field of view of the guest user is divided into the plurality of system-defined volumes, wherein each system-defined volume spans a different three-dimensional portion of the field of view, and wherein at least one of a size or a shape of the volumes is determined based on weighted factors that include an application most recently utilized by the guest user;
determine that the estimated fixation point is within a particular one of the plurality of system-defined volumes that includes multiple depth planes of the plurality of depth planes;
responsive to determining that the system-defined volume includes at least two virtual objects of the virtual image content to be presented, switch the virtual image content, including the at least two virtual objects, to be presented at a depth plane that corresponds to the estimated fixation point, and
responsive to determining that the system-defined volume includes a single virtual object of the virtual image content to be presented, switch the virtual image content, including the single virtual object, to be presented at a depth plane that is specified by information included in the virtual image content.

2. The augmented reality display system of claim 1 , wherein the imaging device comprises a left eye tracking system and a right eye tracking system together configured to measure the wearer's interpupillary distance.

3. The augmented reality display system of claim 1 , wherein the processor is configured to determine whether the wearer is a calibrated user or a guest user based on the wearer's interpupillary distance.

4. The augmented reality display system of claim 3 , wherein the processor is configured to determine whether the wearer is a calibrated user or a guest user based on whether the wearer's interpupillary distance is within a predetermined threshold of the interpupillary distance of the calibrated user.

5. The augmented reality display system of claim 1 , wherein the processor is configured to determine whether the wearer is a calibrated user or a guest user based on whether the wearer's interpupillary distance is within 1.0 mm of the interpupillary distance of the calibrated user.

6. The augmented reality display system of claim 1 , wherein the waveguide is one of a plurality of waveguides forming a stack of waveguides.

7. The augmented reality display system of claim 6 , wherein one or more waveguides of the stack of waveguides are configured to output light to the eye of the wearer with a different amount of wavefront divergence than one or more other waveguides of the stack of waveguides, wherein the outputted light with different amounts of wavefront divergence forms virtual objects at different perceived depths away from the wearer.

8. A method for depth plane switching in a display system configured to direct image light to eyes of a user to display virtual image content, the eyes separated by an interpupillary distance and the display system configured to present the virtual image content on a plurality of depth planes, the method comprising:
determining whether the user is a calibrated user or a guest user;
based on determining that the user is a calibrated user:
loading pre-existing user depth plane switching calibration information;
identifying a fixation point of the user based upon the pre-existing user depth plane switching calibration information; and
switching the virtual image content to be presented at a depth plane that corresponds to the fixation point; and

based on determining that the wearer is a guest user:
determining the interpupillary distance of the guest user;
calculating an estimated fixation point of the eyes of the guest user based upon the determined interpupillary distance, wherein the estimated fixation point is a point in three-dimensional (3D) space on which the eyes of the guest user are focused;
determining a plurality of system-defined volumes such that a field of view of the guest user is divided into the plurality of system-defined volumes, wherein each system-defined volume spans a different three-dimensional portion of the field of view, and wherein at least one of a size or a shape of the volumes is determined based on weighted factors that include an application most recently utilized by the guest user;
determining that the estimated fixation point is within a particular one of the plurality of system-defined volumes that includes multiple depth planes of the plurality of depth planes;
responsive to determining that the system-defined volume includes at least two virtual objects of the virtual image content to be presented, switching the virtual image content, including the at least two virtual objects, to be presented at a depth plane that corresponds to the estimated fixation point, and
responsive to determining that the system-defined volume includes a single virtual object of the virtual image content to be presented, switching the virtual image content, including the single virtual object, to be presented at a depth plane that is specified by information included in the virtual image content.

9. The method of claim 8 , wherein determining the interpupillary distance of the guest user comprises determining an interpupillary distance of eyes of the guest user focused at optical infinity.

10. The method of claim 8 , wherein determining whether the user is the calibrated user or is a guest user comprises determining the interpupillary distance of the user with one or more eye tracking cameras configured to image the eyes of the user.

11. The method of claim 8 , wherein determining whether the user is the calibrated user comprises determining the interpupillary distance of the user with one or more eye tracking cameras configured to image the eyes of the user and determining that the interpupillary distance of the user is within a predetermined range.

12. The method of claim 11 , wherein determining whether the user is the guest user comprises determining the interpupillary distance of the user with the one or more eye tracking cameras and determining that the interpupillary distance of the user is outside of the predetermined range.

13. The method of claim 8 , wherein the pre-existing user depth plane switching calibration information comprises a measured

Assignees

• Magic Leap Inc

Inventors

• Miller Samuel A
• Agarwal Lomesh
• Edwin Lionel Ernest
• Yeoh Ivan Li Chuen
• Farmer Daniel
• Prokushkin Sergey Fyodorovich
• Munk Yonatan
• Selker Edwin Joseph
• Fonseka Erik
• Greco Paul M
• Sommers Jeffrey Scott
• Stuart Bradley Vincent
• Das Shiuli
• Shanbhag Suraj Manjunath

Classifications

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B27/0172 Primary

  characterised by optical features · CPC title

• G02B6/0076

  Stacked arrangements of multiple light guides of the same or different cross-sectional area · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title