Eye tracker illumination through a waveguide

Patent metadata

Field	Value
Publication number	US-11669159-B2
Application number	US-202117208880-A
Country	US
Kind code	B2
Filing date	Mar 22, 2021
Priority date	Mar 22, 2021
Publication date	Jun 6, 2023
Grant date	Jun 6, 2023

Abstract

In a see-through waveguide-based HMD device configured to display holographic virtual images within a field of view (FOV) of the device user, a single pixel or group of pixels are lit to supply illumination at known locations on the display that is reflected from the user's eyes and captured by one or more sensors in an eye tracker. The eye tracker may apply real-time image analysis to the captured reflected light, called “glints,” to extract features of the user's eyes to determine where the HMD device user is looking—the gaze point—and calculate eye movement, location, and orientation. A negative lens functionality utilized in the HMD device to provide a fixed focal depth for the virtual images enables the lit pixels to function as virtual glint sources for the eye tracker sensor that are located at the fixed focal depth and from multiple illumination positions within the user's FOV.

First claim

What is claimed:

1. An illumination system disposed in a near-eye mixed-reality display system adapted for providing illumination for an eye tracker, comprising:
at least one see-through optical waveguide through which real-world images are viewable by a user of the near-eye mixed-reality display system;
an imager generating optical beams for virtual images for each individual color component in a color model utilized in the near-eye mixed reality display system;
an input coupler disposed on the see-through optical waveguide configured to in-couple one or more optical beams for the virtual images into the see-through optical waveguide;
an output coupler disposed on the see-through optical waveguide configured to out-couple the one or more optical beams for the virtual images from the see-through optical waveguide to an eye of a user of the near-eye mixed-reality display system, in which virtual images associated with the out-coupled beams are rendered within a field of view (FOV) of the near-eye mixed reality display system;
a function generator operatively coupled to the imager and configured to cause the imager to light one or more pixels in one or more of the color components at locations within the FOV to provide one or more glint sources for illumination of features of the user's eye, wherein the eye tracker is configured to track one or more of motion, orientation, or gaze direction of the user's eye based on the illuminated features; and
a negative lens disposed on an eye side of the at least one see-through optical waveguide, wherein the negative lens imparts a virtual point of focus to the one or more glint sources at a fixed focal depth that is a non-infinite distance away from the near-eye mixed-reality display system.

2. The illumination system of claim 1 in which the function generator is configured to cause the imager to light one or more pixels that are located in one or more areas of peak intensity within the FOV.

3. The illumination system of claim 1 in which the function generator is configured to cause the imager to light, on the display system, one of a single pixel, a group of adjacent pixels, or a plurality of non-adjacent pixels.

4. The illumination system of claim 3 in which the function generator is configured to cause the imager to light, on the display system, pixels of a single color component of the color model.

5. The illumination system of claim 3 in which the function generator is configured to cause the imager to light, on the display system, pixels for all components of the color model.

6. The illumination system of claim 1 in which the imager renders virtual images on the display system at a frame rate, and in which the function generator is configured to cause the imager to light pixels in one of a single frame, group of successive frames, or a plurality of non-successive frames.

7. The illumination system of claim 1 in which the function generator is operatively coupled to the eye tracker to coordinate illumination of the eye features with capturing of glints from the eye features by the eye tracker.

8. A head-mounted display (HMD) device wearable by a user and supporting a mixed-reality experience including viewing virtual images from a virtual world that are combined with real-world images of objects in a physical world, comprising:
a see-through waveguide-based display system through which the user can view the physical world and on which the virtual images are rendered within a field of view (FOV) of the display system;
an imager generating one or more optical beams for the virtual images that are rendered by the see-through waveguide-based display system, the imager further configured to light one or more individual pixels within the FOV to provide glint sources;
one or more sensors configured to capture glints reflected from features of an eye of the user for eye tracking; and
a negative lens disposed on an eye side of the see-through waveguide-based display system, wherein the negative lens is configured to impart a virtual point of focus to the glint sources at a fixed focal depth that is a non-infinite distance away from the see-through waveguide-based display system.

9. The HMD device of claim 8 in which the virtual images are rendered according to an RGB (red, green, blue) color model and the see-through waveguide-based display system comprises one or more waveguides through which optical beams corresponding to different components of the color model propagate.

10. The HMD device of claim 9 in which the one or more waveguides each include diffractive optical elements respectively configured as an input coupler and an output coupler, the input coupler configured to in-couple one or more optical beams for the virtual images into a respective one or more waveguides, and the output coupler configured to out-couple one or more optical beams for the virtual images from a respective one or more waveguides to an eye of the user, in which virtual images associated with the out-coupled beams are rendered within the FOV.

11. The HMD device of claim 8 in which the see-through waveguide-based display system comprises one or more waveguides that include an input coupler and an output coupler implemented using wavelength-sensitive reflective coatings, in which the input coupler is configured to in-couple one or more optical beams for the virtual images into the one or more waveguides and the output coupler is configured to out-couple the one or more optical beams for the virtual images from the one or more waveguides to an eye of the user, in which virtual images associated with the out-coupled beams are rendered within the FOV.

12. The HMD device of claim 8 further comprising a positive lens and a negative lens that are operable as a conjugate pair, the positive lens disposed on a real-world side of the see-through waveguide-based display system and the negative lens disposed on an eye side of the see-through waveguide-based display system, wherein the negative lens is configured to impart virtual focus to the virtual images at a non-infinite distance from the HMD device.

13. The HMD device of claim 12 in which the negative lens is incorporated into an output coupler disposed on a waveguide in the see-through waveguide-based display system.

14. The HMD device of claim 12 in which the negative lens further imparts virtual focus to the glint sources within the FOV at the non-infinite distance from the HMD device.

15. The HMD device of claim 8 further comprising an IR (infrared) light source that is coupled to and propagating through the see-through waveguide-based display system in an IR light path to provide one or more IR pixels in the FOV, the IR pixels being lit to provide IR glint sources for the eye tracking.

16. The HMD device of claim 15 in which the virtual images are rendered according to an RGB (red, green, blue) color model and the see-through waveguide-based display system comprises a waveguide configured to propagate a red light component of the color model and the IR light.

17. The HMD device of claim 8 in which the imager dynamically lights pixels as glint sources using a variably configurable pattern of pixels or groups of pixels.

18. A method for operating a near-eye optical display system to display virtual images within a field of view (FOV), comprising:
operating an imager in the near-eye optical display system to generate pixels of light;
providing a waveguide having an input coupler configured to in-couple the light pixels from the imager and an output coupler configured to out-couple the pixe

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Reshidko Dmitry
• Wan Congshan

Classifications

• G06V10/60

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06V40/19

  Sensors therefor · CPC title

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G02B2027/0174

  holographic · CPC title

• G02B27/0081

  with means for altering, e.g. enlarging, the entrance or exit pupil · CPC title