Eye tracking system with single point calibration

Patent metadata

Field	Value
Publication number	US-10115205-B2
Application number	US-201715456383-A
Country	US
Kind code	B2
Filing date	Mar 10, 2017
Priority date	Mar 11, 2016
Publication date	Oct 30, 2018
Grant date	Oct 30, 2018

Abstract

A head mounted display (HMD) comprises an eye tracking system configured to perform a calibration process using an eye tracking system of the HMD that includes determining a pupillary axis and/or determining an angular offset between the pupillary axis and the eye's true line of sight. The eye tracking system obtains an eye model captures images of the user's pupil while the user is looking at a target or other content displayed on the HMD. In some embodiments, the calibration process is based on a single image of the user's eye and is performed only once. For example, the process can be performed the first time the user uses the HMD, which stores the calibration data for the user in a memory for future use.

First claim

What is claimed is:

1. A method comprising:
obtaining cornea information for an eye of a user, the cornea information including a radius of a corneal sphere of the eye and location of a center of the corneal sphere;
capturing one or more reflections of a pupil of the eye of the user while the user is viewing content displayed on a head-mounted display (HMD);
determining a shape of the pupil based on the one or more captured pupil reflections;
identifying a three-dimensional plane parallel to the pupil, the three-dimensional plane identified based on the obtained cornea information and the identified pupil shape;
identifying a ray originating from the three-dimensional plane and perpendicular to a surface of the corneal sphere;
determining a pupillary axis for the pupil of the user based on the identified ray; and
performing an optical action based in part on the determined pupillary axis.

2. The method of claim 1 , further comprising:
identifying a foveal centralis region of the eye of the user;
determining a second ray originating from the foveal centralis region of the eye and passing through the pupil corresponding to a true line of sight of the user; and
determining an angular offset between the determined pupillary axis for the pupil of the user and the true line of sight of the user.

3. The method of claim 1 , wherein the cornea information for the eye of the user is obtained by:
capturing one or more reflections of a corneal surface of the eye of the user, the one or more reflections captured while the corneal surface is reflecting light originating from at least two illumination sources positioned at known locations; and
updating a model of the eye including the radius of the corneal sphere of the eye and the location of the center of the corneal sphere of the eye, the model determined based on the captured one or more reflections of the corneal surface and the known locations of the at least two illumination sources.

4. The method of claim 1 , wherein determining the shape of the pupil based on the one or more captured pupil reflections includes:
determining, from the one or more captured pupil reflections, a border between the pupil and an iris of the eye based on a change in reflected intensity of one or more wavelengths of light; and
identifying the shape of the pupil from the determined border.

5. The method of claim 1 , further comprising:
modifying the pupil shape for the user to correct for refractive distortion caused by the corneal sphere of the user.

6. The method of claim 1 , wherein the optical action is selected from the group comprising: determining a gaze direction of the user, determining a vergence angle of the eye of the user, determining the vergence depth of the user, determining an accommodation depth for the user, performing user identification, determining a torsional state of the eye of the user, or some combination thereof.

7. The method of claim 1 , wherein the content displayed by the HMD being viewed by the user includes a known calibration target displayed on an electronic display of the HMD.

8. A head mounted display (HMD) comprising:
an eye tracking system including:
two or more illumination sources configured to illuminate an eye of a user,
a detector configured to capture light reflected from at least one of a cornea or a pupil of the eye of the user as an image, and
a control module configured to:
obtain cornea information for an eye of a user, the cornea information including a radius of a corneal sphere of the eye and location of a center of the corneal sphere,
illuminate the eye of the user with light from the two or more illumination sources,
capture, at the detector, one or more reflections of the pupil of the eye of the user while the user is viewing content displayed by the HMD,
determine a shape of the pupil based on the one or more captured pupil reflections;
identify a three-dimensional plane parallel to the pupil, the three-dimensional plane identified based on the obtained cornea information and the identified pupil shape;
identify a ray originating from the three-dimensional plane and perpendicular to a surface of the corneal sphere;
determine a pupillary axis for the pupil of the user based on the identified ray; and
perform an optical action based in part on the determined pupillary axis.

9. The HMD of claim 8 , wherein the cornea information for the eye of the user is obtained by:
capturing one or more reflections of a corneal surface of the eye of the user, the one or more reflections captured while the corneal surface is reflecting light originating from at least two illumination sources positioned at known locations; and
updating a model of the eye including the radius of the corneal sphere of the eye and the location of the center of the corneal sphere of the eye, the model determined based on the captured one or more reflections of the corneal surface and the known locations of the at least two illumination sources.

10. The HMD of claim 8 , wherein the two or more illumination sources are part of the electronic display.

11. The HMD of claim 8 , wherein determining the shape of the pupil based on the one or more captured pupil reflections includes:
determining, from the one or more captured pupil reflections, a border between the pupil and an iris of the eye based on a change in reflected intensity of one or more wavelengths of light; and
identifying the shape of the pupil from the determined border.

12. The HMD of claim 8 , wherein performing the one or more optical actions comprises performing an action selected from a group consisting of: determining a user's vergence angle, determining a user's accommodation depth, identifying the user, determining an eye's torsional state, or some combination thereof.

13. The HMD of claim 8 , wherein the control module is further configured to:
identify a foveal centralis region of the eye of the user;
determine a second ray originating from the foveal centralis region of the eye and passing through the pupil corresponding to a true line of sight of the user; and
determine an angular offset between the determined pupillary axis for the pupil of the user and the true line of sight of the user.

14. A head mounted display (HMD) comprising:
an electronic display configured to display images to the user;
an optics block configured to magnify light received by the optics block from the electronic display; and
an eye tracking system including:
two or more illumination sources configured to illuminate an eye of a user,
a detector configured to capture light reflected from at least one of a cornea or a pupil of the eye of the user as an image, and
a control module configured to:
obtain cornea information for an eye of a user, the cornea information including a radius of a corneal sphere of the eye and location of a center of the corneal sphere,
illuminate the eye of the user with light from the two or more illumination sources,
capture, at the detector, one or more reflections of the pupil of the eye of the user while the user is viewing content displayed by the HMD,
determine a shape of the pupil based on the one or more captured pupil reflections;
identify a three-dimensional plane parallel to the pupil, the three-dimensional plane identified based on the obtained cornea information and the identified pupil shape;
identify a ray originating from the three-dimensional plane and perpendicular to a surface of the corneal sphere;
determine a pupillary axis for the pupil of the user based on the identified ray;
identify a foveal centralis region of the eye of the user;
determi

Assignees

• Oculus Vr Llc
• Facebook Tech Llc

Inventors

• Cavin Robert Dale
• Hunt Warren Andrew

Classifications

• G06T7/60

  Analysis of geometric attributes · CPC title

• G06T2207/10152

  Varying illumination · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title

• G06T7/564

  from contours · CPC title

• G06T7/75

  involving models · CPC title