Three-dimensional (3D) compressive sensing based eye tracking

Patent metadata

Field	Value
Publication number	US-12355216-B2
Application number	US-202218087322-A
Country	US
Kind code	B2
Filing date	Dec 22, 2022
Priority date	Dec 22, 2022
Publication date	Jul 8, 2025
Grant date	Jul 8, 2025

Abstract

A three-dimensional (3D) compressive sensing based eye tracking system using single photon avalanche diode (SPAD) sensors achieves high resolution depth measurement by using low resolution SPAD sensors and active encoded illumination such as two- or three-dimensional fringe patterns, random speckles, random patterns, and/or superimposed patterns projected onto a surface of an eye. The patterns may be projected by high speed illuminators such as a digital micromirror device (DMD) or micro-electromechanical system (MEMS) projector in series changing at the same rate as the SPAD sensor capture rate. A processor may employ compressive sensing techniques to obtain a high resolution image and depth information from the captured images of varying patterns.

First claim

The invention claimed is:

1. An eye tracking system, comprising:
a projector to project a sequence of active encoded illumination patterns onto a surface of an eye of a user;
a single-photon avalanche diode (SPAD) based sensor system to capture reflections of the active encoded illumination patterns from the surface of the eye; and
a processor to:
synchronize a rate of the projector projecting the active encoded illumination patterns with a rate of the SPAD based sensor system capturing the reflections of the active encoded illumination patterns;
employ compressive sensing to process the captured reflections of the active encoded illumination patterns to obtain high resolution images from the captured reflections; and
determine three-dimensional (3D) features of the surface of the eye from the high resolution images, wherein the 3D features of the surface of the eye are used to determine a gaze of the user.

2. The eye tracking system of claim 1 , wherein the projector is to project the active encoded illumination patterns onto the surface of the eye at a rate of up to 500 KHz.

3. The eye tracking system of claim 1 , wherein the SPAD based sensor system comprises at least one SPAD sensor, a linear SPAD sensor array, or a two-dimensional SPAD sensor array.

4. The eye tracking system of claim 1 , wherein the projector comprises:
a laser source to provide a collimated light; and
a micro-electromechanical system (MEMS) reflector to project the active encoded illumination patterns at the specific rate by directing the collimated light.

5. The eye tracking system of claim 4 , wherein the processor is to synchronize the rate of SPAD based sensor system capturing the reflections of the active encoded illumination patterns with the specific rate of the MEMS reflector projecting the active encoded illumination patterns.

6. The eye tracking system of claim 4 , wherein the MEMS reflector operates at the specific rate of 500 KHz.

7. The eye tracking system of claim 1 , wherein the projector comprises:
a light source to provide light; and
a digital micromirror device (DMD) reflector to project the active encoded illumination patterns by directing the light from the light source.

8. The eye tracking system of claim 7 , wherein the light source comprises a side-emitting laser diode, a vertical-cavity surface-emitting laser (VCSEL) diode, a superluminescent light-emitting diode (SLED), or a light-emitting diode (LED).

9. The eye tracking system of claim 1 , wherein the active encoded illumination patterns comprise two-dimensional encoded patterns, speckled patterns, random patterns, and combinations thereof.

10. The eye tracking system of claim 9 , wherein a number of the active encoded illumination patterns is selected based on at least one of a pattern complexity, a projector type, and a sensor system type.

11. A method for operating an eye tracking system, comprising:
projecting, by a projector in the eye tracking system, a sequence of active encoded illumination patterns onto a surface of an eye of a user at a specific rate;
capturing, by a single-photon avalanche diode (SPAD) based sensor system in the eye tracking system, reflections of the active encoded illumination patterns from the surface of the eye at a same rate as the specific rate of the projector projecting the active encoded illumination patterns;
employing compressive sensing, by a processor in the eye tracking system, to process the captured reflections of the active encoded illumination patterns to obtain high resolution images from the captured reflections;
determining, by the processor, three-dimensional (3D) features of the surface of the eye from the high resolution images; and
determining, by the processor, a gaze of the user based on the 3D features of the surface of the eye.

12. The method of claim 11 , wherein projecting the sequence of the active encoded illumination patterns onto the surface of the eye comprises:
projecting deterministic patterns or random patterns onto the surface of the eye.

13. The method of claim 12 , wherein
the deterministic patterns comprise orthogonal patterns or two-dimensional encoded patterns, and
the random patterns comprise speckled patterns or pseudo-random patterns.

14. The method of claim 11 , further comprising:
selecting a number of the active encoded illumination patterns based on at least one of a pattern complexity, a projector type, and a sensor system type.

15. The method of claim 11 , wherein projecting the sequence of the active encoded illumination patterns comprises:
providing a collimated light from a laser source to a micro-electromechanical system (MEMS) reflector; and
projecting the active encoded illumination patterns at the specific rate by directing the collimated light from the MEMS reflector onto the surface of the eye,
wherein the SPAD based sensor system is to capture the reflections of the active encoded illumination patterns at the same rate as the specific rate of the MEMS reflector projecting the active encoded illumination patterns.

16. The method of claim 11 , wherein projecting the sequence of the active encoded illumination patterns comprises:
providing light from a light source to a digital micromirror device (DMD) reflector; and
projecting the sequence of the active encoded illumination patterns by directing the light from the light source from the DMD reflector onto the surface of the eye.

17. A non-transitory computer-readable storage medium storing machine readable instructions which when executed by a processor cause the processor to:
cause a projector in an eye tracking system to project a sequence of active encoded illumination patterns onto a surface of an eye at a specific rate;
cause a single-photon avalanche diode (SPAD) based sensor system in the eye tracking system to capture reflections of the active encoded illumination patterns from the surface of the eye at a same rate as the specific rate of the projector projecting the active encoded illumination patterns;
employ compressive sensing to process the captured reflections of the active encoded illumination patterns to obtain high resolution images from the captured reflections;
determine three-dimensional (3D) features of the surface of the eye from the high resolution images; and
determine a gaze of the user based on the 3D features of the surface of the eye.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the machine readable instructions cause the processor to select a number of the active encoded illumination patterns based on at least one of a pattern complexity, a projector type, and a sensor system type.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the active encoded illumination patterns comprise orthogonal deterministic patterns or random patterns.

20. The non-transitory computer-readable storage medium of claim 17 , wherein the projector comprises a micro-electromechanical system (MEMS) reflector to project the active encoded illumination patterns onto the surface of the eye at the specific rate, and
wherein the instructions cause the processor to synchronize the rate of SPAD based sensor system capturing the reflections of the active encoded illumination patterns with the specific rate of the MEMS reflector projecting the active encoded illumination patterns.

Assignees

• Meta Platforms Tech Llc

Inventors

• An Yatong
• Wang Youmin
• Nie Zhaoyu

Classifications

• G01B11/2513

  with several lines being projected in more than one direction, e.g. grids, patterns · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G02B27/017

  Head mounted · CPC title

• H01S5/423 Primary

  having a vertical cavity · CPC title

• G02B27/0093 Primary

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title