Eye tracking systems and method for augmented or virtual reality

Patent metadata

Field	Value
Publication number	US-10825248-B2
Application number	US-201514707000-A
Country	US
Kind code	B2
Filing date	May 8, 2015
Priority date	Apr 18, 2014
Publication date	Nov 3, 2020
Grant date	Nov 3, 2020

Abstract

An augmented reality display system comprises a passable world model data comprises a set of map points corresponding to one or more objects of the real world. The augmented reality system also comprises a processor to communicate with one or more individual augmented reality display systems to pass a portion of the passable world model data to the one or more individual augmented reality display systems, wherein the piece of the passable world model data is passed based at least in part on respective locations corresponding to the one or more individual augmented reality display systems.

First claim

What is claimed is:

1. A method, comprising:
rendering one or more virtual objects in a virtual or augmented reality environment with a virtual image generation subsystem of a virtual content display system altering, with a variable focus element, one or more rays or cones of light among multiple planes that correspond to respective virtual depths towards at least one eye of a user, wherein
the virtual image generation subsystem comprising a graphics processing unit, at least one projector, the variable focus element, a first assembly, and a second assembly that is operatively coupled to the first assembly,
the first assembly and second assembly receive power from a portable power source separate from the virtual image generation subsystem,
the first assembly includes a first board and a projector driver providing image information and signals to the at least one projector,
the second assembly includes a second board, a microprocessor, the graphics processing unit, and one or more motion sensors or transducers, and
the at least one projector projects image information of the one or more virtual objects to the at least one eye of the user;

representing the at least one eye with an eye model that comprises a first circular shape or circle and a second circular shape or circle, wherein the second circular shape or circle represents a cornea of the at least one eye and is layered on top of the first circular shape or circle;
detecting, with at least an eye tracking device, one or more characteristics pertaining to an interaction between the at least one eye of the user and reflected light from the at least one eye at least by:
capturing, by a first set of sensors or transducers in the first assembly, a first pattern emitted or reflected from one or more ambient light sources in a real-world environment;
projecting, with at least the projector driver in the first assembly and the at least one projector, a second light pattern generated by a set of light sources in the first assembly to the at least one eye of the user;
in response to the first light pattern, detecting the reflected light from the at least one eye using one or more second sensors or transducers in the first assembly; and
determining, by the microprocessor in the second assembly, the interaction at least by correlating the reflected light with the first light pattern and the second light pattern;

determining, by the microprocessor in the second assembly, an eye pointing vector and a center of rotation of the at least one eye in the eye model using at least the first circular shape or circle and the second circular shape or circle based at least in part upon a characteristic of a cross-section of the at least one eye and a range of movement of the at least one eye;
determining, by the microprocessor in the second assembly, a vectored distance for the eye pointing vector based at least in part upon the one or more characteristics and the center of rotation for the at least one eye in the eye model; and
determining, by the microprocessor in the second assembly, at least one movement or pose for both the at least one eye and another eye of the user at least by using the vectored distance of the at least one eye and further by extrapolating one or more eye movement or pose characteristics with at least one or more parameters pertaining to the interaction and captured by one or more sensors for the at least one eye of the user.

2. The method of claim 1 , wherein at least one characteristic of the one or more characteristics pertains to light reflected back from the at least one eye, and the plurality of eye pointing vectors are determined without using cameras.

3. The method of claim 1 , wherein at least one characteristic of the one or more characteristics pertains to one or more reflections of one or more objects from a structure of the at least one eye of the user, and the one or more characteristics comprise an extent of backscattered light detected by the first set of sensors or transducers in the first assembly.

4. The method of claim 1 , further comprising varying at least one parameter of the one or more rays or cones of light.

5. The method of claim 4 , further comprising pseudo-randomly varying the at least one parameter.

6. The method of claim 4 , wherein the at least one parameter corresponds to a length of emission of the light source.

7. The method of claim 1 , wherein the one or more rays or cones of light are emitted in a predetermined pattern.

8. The method of claim 7 , further comprising correlating the one or more characteristics with a set of one or more known characteristics to determine the at least one movement or pose.

9. The method of claim 1 , further comprising determining the at least one movement or pose based at least in part on a known distance of the at least one eye from one or more sensors detecting a first characteristic of the interaction between the one or more rays or cones of light and the at least one eye of the user and a plurality of light sources emitting the one or more rays or cones of light to the at least one eye of the user.

10. The method of claim 1 , further comprising:
configuring a heat map that tracks and records at least one or more numbers of instances of poses or movements of the at least one eye of the user in relation to one or more virtual objects or one or more real-world objects;
storing information of at least one virtual object or at least one real-world object that produces a most number of instances of the poses or the movements in the heat map; and
rendering virtual contents to the user based in part or in whole upon the heat map or upon the one or more characteristics pertaining to the interaction.

11. A virtual content display system for tracking one or more movements or poses of a user's eyes, comprising:
a virtual image generation subsystem configured to rendering one or more virtual objects in a virtual or augmented reality environment with a virtual image generation subsystem of a virtual content display system altering, with a variable focus element, one or more rays or cones of light among multiple planes that corresponds to respective virtual depths towards at least one eye of a user, wherein
the virtual image generation subsystem comprising a graphics processing unit, at least one projector, the variable focus element, a first assembly, and a second assembly that is operatively coupled to the first assembly,
the first assembly and second assembly receive power from a portable power source separate from the virtual image generation subsystem,
the first assembly includes a first board and a projector driver providing image information and signals to the at least one projector,
the second assembly includes a second board, a microprocessor, the graphics processing unit, and one or more motion sensors or transducers, and
the at least one projector projects image information of the one or more virtual objects to the at least one eye of the user;

an eye model that represents the at least one eye with a first circular shape or circle and a second circular shape or circle that models a cornea of the at least one eye and is layered on top of the first circular shape or circle;
an eye tracking device positioned in relation to the virtual image generation subsystem as well as the at least one eye of the user and configured at least to:
detect one or more characteristics pertaining to an interaction between the at least one eye of the user and reflected light from the at least one eye at least by:
capturing, by a first set of sensors or transducers in the first assembly, a first pattern emitted or reflected from one or more ambient light sources in a real-w

Assignees

• Magic Leap Inc

Inventors

• Miller Samuel A

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06V20/653

  by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06V20/80

  Recognising image objects characterised by unique random patterns · CPC title