Dynamic eye tracking calibration

Patent metadata

Field	Value
Publication number	US-9851791-B2
Application number	US-201514940658-A
Country	US
Kind code	B2
Filing date	Nov 13, 2015
Priority date	Nov 14, 2014
Publication date	Dec 26, 2017
Grant date	Dec 26, 2017

Abstract

A user of a computing device may interact with and control objects and applications displayed on the computing device through the user's eye movement. Detected gaze locations are correlated with actions performed by the user and compared with typical gaze locations for those actions. Based on differences between the detected and expected gaze locations, the eye tracking system can be recalibrated. An area around a gaze location encompassing a set of likely active locations can be enlarged, effectively prompting the user to interact with the desired active location again. The enlarging of the area serves to separate the active locations on the screen, reducing the probability of interpreting the user's gaze incorrectly.

First claim

What is claimed is:

1. A device comprising:
a display;
a memory having instructions embodied thereon; and

one or more processors configured by the instructions to perform operations comprising:
maintaining a buffer with gaze information of a user of a computing device, the gaze information including one or more elements of data for each gaze capture, the one or more elements of data identifying an estimated gaze location that provides an estimation of a position on the display where the user is looking at a timestamp, the one or more elements of data being determined for each gaze capture by an eye tracking module at the timestamp based on light captured by a camera of the eye tracking module after reflection from an eye of the user;
during execution of an application on the computing device, detecting a user interaction at an interaction location on the display of the computing device, the user interaction occurring at an interaction time;
in response to the detection of the user interaction, selecting a portion of the gaze information from the buffer, the selected portion comprising information about one or more gaze captures occurring within a threshold time period before the interaction time;
based on the portion of the gaze information and the interaction location, calculating one or more calibration parameters specific to the user including at least one of: offsets between optical and visual axes of the eye, a radius of a cornea of the eye, a distance between a center of the cornea and a center of a pupil of the eye;
determining, by the eye tracking module, eye feature information including components of a vector difference between the center of the pupil and a corneal reflection of the eye; and
computing a point of regard of the user on the display based on the eye feature information and the one or more calibration parameters.

2. The device of claim 1 , wherein the operations further comprise:
clustering the portion of gaze information into a set of clusters of gaze fixations of the user on the display;
selecting a cluster from the set of clusters based on distances of the gaze fixations from the interaction location; and
calculating the one or more calibration parameters based on the selected cluster.

3. The device of claim 1 , wherein the operations further comprise:
clustering the portion of the gaze information into a set of clusters of gaze fixations of the user on the display;
selecting a cluster from the set of clusters based on a quality metric; and
calculating the one or more calibration parameters based on the selected cluster.

4. The device of claim 3 , wherein the quality metric is based on a factor selected from the group consisting of a distance of a gaze fixation in the cluster from the interaction location, a time of the gaze fixation relative to the interaction time, and intra-cluster coherence.

5. The device of claim 1 , wherein:
the one or more elements of data for each gaze capture in the selected portion of the gaze information includes one or more eye features estimated by the eye tracking module and the estimated gaze location; and the operations further comprise:
storing information about the one or more eye features and the estimated gaze location into a calibration data structure; and
calculating the one or more calibration parameters based on the information stored in the calibration data structure.

6. The device of claim 5 , wherein the operations further comprise:
computing a quality metric of at least one element of data in the selected portion of the gaze information associated with the information stored in the calibration data structure;
removing low-quality elements from the calibration data structure based on the quality metric; and
calculating the one or more calibration parameters based on remaining information stored in the calibration data structure after removing the low-quality elements.

7. The device of claim 6 , wherein the quality metric is based on a location on the display associated with the at least one element relative to the interaction location.

8. The device of claim 6 , wherein the quality metric is based on a time elapsed since the timestamp when the one or more eye features were captured by the eye tracking module.

9. The device of claim 6 , wherein the quality metric is based on an estimation error associated with the estimated gaze location.

10. A method comprising:
maintaining a buffer with gaze information of a user of a computing device, the gaze information including one or more elements of data for each gaze capture, the one or more elements of data identifying an estimated gaze location that provides an estimation of a position on a display of the computing device where the user is looking at a timestamp, the one or more elements of data being determined for each gaze capture by an eye tracking module at the timestamp based on light captured by a camera of the eye tracking module after reflection from an eye of the user;
during execution of an application on the computing device, detecting a user interaction at an interaction location on the display of the computing device, the user interaction occurring at an interaction time;
in response to the detection of the user interaction, selecting a portion of the gaze information from the buffer, the selected portion comprising information about one or more gaze captures occurring within a threshold time period before the interaction time;
based on the portion of the gaze information and the interaction location, calculating one or more calibration parameters specific to the user including at least one of: offsets between optical and visual axes of the eye, a radius of a cornea of the eye, a distance between a center of the cornea and a center of a pupil of the eye;
determining, by the eye tracking module, eye feature information including components of a vector difference between the center of the pupil and a corneal reflection of the eye; and
computing a point of regard of the user on the display based on the eye feature information and the one or more calibration parameters.

11. The method of claim 10 , further comprising:
clustering the portion of gaze information into a set of clusters of gaze fixations of the user on the display;
selecting a cluster from the set of clusters based on distances of the gaze fixations from the interaction location; and
calculating the one or more calibration parameters based on the selected cluster.

12. The method of claim 10 , further comprising:
clustering the portion of the gaze information into a set of clusters of gaze fixations of the user on the display;
selecting a cluster from the set of clusters based on a quality metric; and
calculating the one or more calibration parameters based on the selected cluster.

13. The method of claim 12 , wherein the quality metric is based on a factor selected from the group consisting of a distance of a gaze fixation in the cluster from the interaction location, a time of the gaze fixation relative to the interaction time, and intra-cluster coherence.

14. The method of claim 10 , wherein:
the one or more elements of data for each gaze capture in the selected portion of the gaze information includes one or more eye features estimated by the eye tracking module and the estimated gaze location; and the method further comprising:
storing information about the one or more eye features and the estimated gaze location into a calibration data structure; and
calculating the one or more calibration parameters based on the information stored in the calibration data structure.

15. Th

Assignees

• Facebook Inc

Inventors

• San Agustin Lopez Javier
• Tall Martin Henrik
• Lefaudeux Benjamin Antoine Georges
• Skovsgaard Henrik Hegner Tomra

Classifications

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/038

  Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06K9/00604

  Physics · mapped topic