Systems, devices, and methods for laser eye tracking in wearable heads-up displays

Patent metadata

Field	Value
Publication number	US-10409057-B2
Application number	US-201715827667-A
Country	US
Kind code	B2
Filing date	Nov 30, 2017
Priority date	Nov 30, 2016
Publication date	Sep 10, 2019
Grant date	Sep 10, 2019

Abstract

Systems, devices, and methods that use elements of a scanning laser projector (“SLP”) to determine the gaze direction of a user of a wearable heads-up display (“WHUD”) are described. An infrared laser diode is added to an RGB SLP and an infrared photodetector is aligned to detect reflections of the infrared light from the eye. A scan mirror in the SLP sweeps through a range of orientations and the intensities of reflections of the infrared light are monitored by a processor to determine when a spectral reflection or “glint” is produced. The processor determines the orientation of the scan mirror that produced the glint and maps the scan mirror orientation to a region in the field of view of the eye of the user, such as a region in visible display content projected by the WHUD, to determine the gaze direction of the user.

First claim

The invention claimed is:

1. A method of determining a gaze direction of an eye of a user, the method comprising:
generating an infrared laser light by an infrared laser diode;
scanning the infrared laser light over the eye of the user by at least one scan mirror, wherein scanning the infrared laser light over the eye of the user by the at least one scan mirror includes sweeping the at least one scan mirror through a range of orientations and, for a plurality of orientations of the at least one scan mirror, reflecting the infrared laser light to a respective region of the eye of the user;
detecting reflections of the infrared laser light from the eye of the user by at least one infrared photodetector;
determining a respective intensity of a plurality of detected reflections of the infrared laser light by at least one processor communicatively coupled to the at least one infrared photodetector;
identifying, by the at least one processor, at least one detected reflection for which the intensity exceeds a threshold value;
determining, by the at least one processor, the orientation of the at least one scan mirror that corresponds to the at least one detected reflection for which the intensity exceeds the threshold value; and
determining, by the at least one processor, a region in a field of view of the eye of the user at which a gaze of the eye is directed based on the orientation of the at least one scan mirror that corresponds to the at least one detected reflection for which the intensity exceeds the threshold value.

2. The method of claim 1 wherein scanning the infrared laser light over the eye of the user by at least one scan mirror includes scanning, by the at least one scan mirror, the infrared laser light over an area of a holographic optical element positioned in the field of view of the eye of the user and redirecting the infrared laser light towards the eye of the user by the holographic optical element.

3. The method of claim 2 wherein redirecting the infrared laser light towards the eye of the user by the holographic optical element includes converging the infrared laser light to an exit pupil at the eye of the user by the holographic optical element.

4. The method of claim 1 wherein scanning the infrared laser light over the eye of the user by at least one scan mirror includes scanning the infrared laser light across a first dimension of the eye of the user by a first scan mirror and scanning the infrared laser light across a second dimension of the eye of the user by a second scan mirror, and wherein:
sweeping the at least one scan mirror through a range of orientations includes sweeping the first scan mirror through a first range of orientations and, for each respective orientation of the first scan mirror, sweeping the second scan mirror through a second range of orientations;
for a plurality of orientations of the at least one scan mirror, reflecting the infrared laser light to a respective region of the eye of the user includes, for a plurality of combinations of a first orientation of the first scan mirror and a second orientation of the second scan mirror, reflecting the infrared laser light to a respective region of the eye of the user;
determining, by the at least one processor, the orientation of the at least one scan mirror that corresponds to the at least one detected reflection for which the intensity exceeds the threshold value includes determining, by the at least one processor, the combination of the first orientation of the first scan mirror and the second orientation of the second scan mirror that corresponds to the at least one detected reflection for which the intensity exceeds the threshold value; and
determining, by the at least one processor, a region in a field of view of the eye of the user at which a gaze of the eye is directed based on the orientation of the at least one scan mirror that corresponds to the at least one detected reflection for which the intensity exceeds the threshold value includes determining, by the at least one processor, the region in the field of view of the eye of the user at which the gaze of the eye is directed based on the combination of the first orientation of the first scan mirror and the second orientation of the second scan mirror that corresponds to the at least one detected reflection for which the intensity exceeds the threshold value.

5. The method of claim 1 wherein identifying, by the at least one processor, at least one detected reflection for which the intensity exceeds a threshold value includes detecting, by the infrared photodetector, a spectral reflection of the infrared laser light from the eye of the user.

6. The method of claim 1 wherein identifying, by the at least one processor, at least one detected reflection for which the intensity exceeds a threshold value includes sampling, by the processor, a signal from the infrared photodetector and identifying, by the processor, a first sample for which the magnitude exceeds a threshold magnitude.

7. The method of claim 6 wherein identifying, by the at least one processor, at least one detected reflection for which the intensity exceeds a threshold value further includes identifying, by the at least one processor, a second sample for which the magnitude does not exceed the threshold magnitude.

8. The method of claim 1 wherein determining, by the at least one processor, a region in a field of view of the eye of the user at which a gaze of the eye is directed based on the orientation of the at least one scan mirror that corresponds to the at least one detected reflection for which the intensity exceeds the threshold value includes effecting, by the at least one processor, a mapping between the orientation of the at least one scan mirror that corresponds to the at least one detected reflection for which the intensity exceeds the threshold value and the field of view of the eye of the user.

9. The method of claim 8 wherein effecting, by the at least one processor, a mapping between the orientation of the at least one scan mirror that corresponds to the at least one detected reflection for which the intensity exceeds the threshold value and the field of view of the eye of the user includes performing, by the at least one processor, at least one transformation selected from a group consisting of: a linear transformation between a set of scan mirror orientations and a set of gaze directions of the eye of the user, a geometric transformation between a set of scan mirror orientations and a set of gaze directions of the eye of the user, an affine transformation between a set of the scan mirror orientations and a set of gaze directions of the eye of the user, and a neural network transformation between a set of scan mirror orientations and a set of gaze directions of the eye of the user.

10. The method of claim 1 wherein the infrared laser diode and the at least scan mirror are components of a scanning laser projector, the scanning laser projector further including at least one additional laser diode to generate visible laser light, and wherein the method further comprises:
projecting visible display content in the field of view of the eye of the user by the scanning laser projector, wherein determining, by the at least one processor, a region in a field of view of the eye of the user at which a gaze of the eye is directed based on the orientation of the at least one scan mirror that corresponds to the at least one detected reflection for which the intensity exceeds the threshold value includes determining, by the at least one processor, a region of the visible display content at which the gaze of the eye is directed based on the orientation of the at least one scan mirror that corresponds to the at l

Assignees

• North Inc

Inventors

• Aleem Idris S
• Bhargava Mayank

Classifications

• G02B2027/0123

  comprising devices increasing the field of view · CPC title

• G02B27/0176

  characterised by mechanical features · CPC title

• G02B27/0103

  comprising holographic elements · CPC title

• H04N9/3102

  using two-dimensional electronic spatial light modulators (micromechanical modulators as such G02B26/0833; liquid crystal modulators as such G02F1/13) · CPC title

• H04N13/383

  for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes · CPC title