Eye tracking based on light polarization

Patent metadata

Field	Value
Publication number	US-10108261-B1
Application number	US-201715642112-A
Country	US
Kind code	B1
Filing date	Jul 5, 2017
Priority date	Jul 5, 2017
Publication date	Oct 23, 2018
Grant date	Oct 23, 2018

Abstract

A head mounted display (HMD) comprises an eye tracking system configured to enable eye-tracking using polarization. The eye tracking system includes an illumination source and an eye tracking unit comprising a polarization sensitive optical detector. The one or more illumination sources are configured to illuminate an eye and generate reflections directed towards the optical detector. The eye tracking unit is configured to determine a 3D shape of the eye based on the polarization of the reflections. The determined 3D shape of the eye is used to update a stored model of the eye in response to the one or more model parameter values extracted from the determined depth map of the corneal surface. The eye tracking system determines eye tracking information based on the updated model in order to improve eye tracking performance.

First claim

What is claimed is:

1. A head-mounted display (HMD) comprising:
an eye tracking system that includes:
one or more illumination sources configured to periodically illuminate an eye of a user with light, and
an optical detector comprising a plurality of polarization sensitive pixels configured to:
capture one or more images of the eye, and
determine intensity information and polarization information associated with the one or more images; and

a controller configured to:
determine a three-dimensional (3D) shape of the eye based on the determined intensity and polarization information,
update a model of the eye based on the determined 3D shape of the eye, and
determine eye tracking information associated with the updated model of the eye, the eye tracking information including an eye orientation of the eye.

2. The HMD of claim 1 , wherein the one or more illumination sources are configured to emit unpolarized light.

3. The HMD of claim 1 , wherein the one or more illumination sources are integrated into a display unit configured to display one or more images.

4. The HMD of claim 1 wherein each of the plurality of polarization sensitive pixels includes three or more native pixels wherein each of the three or more native pixels comprises a CMOS detector coupled to a polarization filter.

5. The HMD of claim 1 , wherein determining a 3D shape of the eye comprises:
determining a plurality of surface normals associated with each of the plurality of polarization sensitive pixels; and
for each of the plurality of the determined surface normals, determining a tangent plane; and
merging each of the determined tangent planes.

6. The HMD of claim 5 wherein determining a plurality of surface normals comprises:
for each of the plurality of polarization sensitive pixels:
capturing, for at least three polarization angles, an intensity of the reflected light,
determining an azimuthal angle associated with the reflected light based on the determined polarization angle and the intensity,
determining a degree of polarization of the reflected light, and
determining a zenith angle based in part on the determined degree of polarization.

7. The HMD of claim 1 , wherein updating the model of the eye based on the determined 3D shape of the eye comprises:
retrieving one or more model parameter values associated with a stored model of the eye;
extracting one or more model parameter values from the determined 3D shape of the eye;
responsive to determining a difference between at least one of the one or more stored model parameters values and the one or more extracted model parameters values, updating at least one of the one or more model parameters by replacing the one or more stored model parameter values with the one or more extracted model parameters values; and
storing the updated model in a data store associated with the eye tracking module.

8. The method of claim 7 , wherein the stored model of the eye comprises a first sphere that describes a shape of a sclera of the eye and a second sphere that describes a shape of a cornea of the eye, the stored model is parameterized by a radius associated with each of the two spheres, a location of the centers of each of the two spheres, and a distance between the centers each of the two spheres.

9. The HMD of claim 1 , wherein the determined eye tracking information includes one or more of a gaze direction, a vergence angle, a vergence depth, and an accommodation depth.

10. The HMD of claim 1 , further comprising:
a display unit configured to emit content; and
an optics block configured to direct the content to an eyebox of the HMD wherein the emitted content is based at least in part on the eye tracking information.

11. The HMD of claim 1 , wherein the determined eye tracking information is utilized to enable one or more of: a gaze-based selection, an attention estimation, one or more gaze-contingent display modes.

12. A head-mounted display (HMD) comprising:
a display unit including one or more illumination sources, the display unit configured to emit content and illuminate an eye of a user,
an eye tracking system that includes:
an optical detector comprising a plurality of polarization sensitive pixels configured to:
capture one or more images of the eye, and
determine intensity information and polarization information associated with the one or more images; and

a controller configured to:
determine a three-dimensional (3D) shape of the eye based on the determined intensity and polarization information,
update a model of the eye based on the determined 3D shape of the eye, and
determine eye tracking information associated with the updated model of the eye, the eye tracking information including an eye orientation of the eye.

13. The HMD of claim 12 , wherein determining a 3D shape of the eye comprises:
determining a plurality of surface normals associated with each of the plurality of polarization sensitive pixels; and
for each of the plurality of the determined surface normals, determining a tangent plane; and
merging each of the determined tangent planes.

14. The HMD of claim 13 wherein determining a plurality of surface normals comprises:
for each of the plurality of polarization sensitive pixels:
capturing, for at least three polarization angles, an intensity of the reflected light,
determining an azimuthal angle associated with the reflected light based on the determined polarization angle and the intensity,
determining a degree of polarization of the reflected light, and
determining a zenith angle based in part on the determined degree of polarization.

15. The HMD of claim 12 , wherein updating a model of the eye based on the determined 3D shape of the eye comprises:
retrieving one or more model parameter values associated with a stored model of the eye;
extracting one or more model parameter values from a depth map;
responsive to determining a difference between at least one of the one or more stored model parameters values and the one or more extracted model parameters values, updating at least one of the one or more model parameters by replacing the one or more stored model parameter values with the one or more extracted model parameters values; and
storing the updated model in a data store associated with the eye tracking module.

16. The method of claim 15 , wherein the stored model of the eye comprises a first sphere that describes a shape of a sclera of the eye and a second sphere that describes a shape of a cornea of the eye, the stored model is parameterized by a radius associated with each of the two spheres, a location of the centers of each of the two spheres, and a distance between the centers each of the two spheres.

17. The HMD of claim 12 , wherein the determined eye tracking information includes one or more of a gaze direction, a vergence angle, a vergence depth, and an accommodation depth.

18. A head-mounted display (HMD) comprising:
an eye tracking system that includes:
one or more illumination sources configured to illuminate an eye of a user with polarized light, and
an optical detector comprising a plurality of polarization sensitive pixels coupled to a dynamic polarizer, each of the polarization sensitive pixels configured to:
capture one or more images of the eye, and
determine intensity information and polarization information associated with the one or more images; and

a controller configured to:
generate a depth map associated with the one or more captured images using the determined intensity and polarization information

Assignees

• Oculus Vr Llc

Inventors

• Hall Michael
• Sharma Robin
• Trail Nicholas Daniel
• Cavin Robert Dale

Classifications

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G01S17/66

  Tracking systems using electromagnetic waves other than radio waves · CPC title

• G02B27/017

  Head mounted · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G02B5/30

  Polarising elements (light-modulating devices with active elements G02F1/00) · CPC title