Eye gaze tracking system and method

Patent metadata

Field	Value
Publication number	US-9411417-B2
Application number	US-201414324758-A
Country	US
Kind code	B2
Filing date	Jul 7, 2014
Priority date	Jul 7, 2014
Publication date	Aug 9, 2016
Grant date	Aug 9, 2016

Abstract

An eye gaze tracking system is disclosed. The system includes a gaze data acquisition system including a plurality of light sources and a plurality of image sensors. The plurality of light sources are arranged to emit light to a head of the user, and the plurality of image sensors are configured to receive the light. In an embodiment, the system further includes a gaze tracking module including an ocular feature extraction module, a point of regard (POR) calculation module and a POR averaging module. The ocular feature extraction module is configured to process the gaze data and to extract ocular features, and is configured to determine a confidence value associated with an accuracy of the parameters. The POR calculation module is configured to determine a POR from the ocular features. The POR averaging module is configured to determine an average POR.

First claim

The invention claimed is:

1. An eve gaze tracking system for determining a point of regard (POR) of a user, the system comprising:
a gaze data acquisition system comprising a plurality of light sources and a plurality of image sensors, the plurality of light sources arranged to emit light to a head of the user, and the plurality of image sensors configured to receive light from the head of the user to capture gaze data thereof:
a gaze tracking module comprising an ocular feature extraction module, a POR calculation module and a POR averaging module,
wherein the ocular feature extraction module is configured to process the gaze data and to extract in relation to each image sensor a set of ocular features, and is further configured to determine, for each set of ocular features related to a captured frame, a confidence value associated with an accuracy of captured parameters; and
wherein the POR calculation module is configured to determine an estimated POR value and the POR averaging module is configured to determine the confidence value of the estimated POR value, said confidence value being proportional to weight factors related to head pose angle of the user and feature detection reliability.

2. The system according to claim 1 , wherein the POR calculation module is configured to determine an estimated POR value from the ocular features related to glint locations and location of pupil center for each image sensor independently.

3. The system according to claim 1 , wherein the POR averaging module is configured to determine a corrected POR value from the confidence value and the estimated POR from each set of ocular features captured for each frame.

4. The system according to claim 3 , wherein the POR averaging module is configured to determine a corrected POR value for each set of ocular features from a weighted average calculation based on the estimated POR's and the confidence values of each estimated POR.

5. The system according to claim 3 , wherein the POR averaging module is configured to determine a corrected POR value for each set of ocular features based on the estimated PORs and the confidence values of each estimated POR, wherein only the corrected POR equals the estimated POR with the highest confidence value.

6. The system according to claim 1 , wherein the eye gaze tracking system is further configured to determine a POR of a user on a display, wherein said display is in communication with a computer device.

7. The system according to claim 1 , wherein the eye gaze tracking system is operationally connected to a computer device and the display, and wherein the determined POR is configured to be received in the computer device, such that a cursor or an item on the screen is following the POR of the user.

8. The system according to claim 1 , wherein the gaze data acquisition system comprises four light sources located, each light source being located in each of four corners of a monitor, and one light source located in the optical axis of each camera sensor.

9. The system according to claim 1 , wherein the image sensor is a camera configured to produce a stream of captured video data comprising the ocular data.

10. A method for determining a point of regard (POR) of a user, using a gaze data acquisition system comprising a plurality of light sources and a plurality of image sensors; said method comprising the steps of:
Detecting a location of the eyes of a user by illuminating a facial region with the light sources and capturing reflected light with each image sensor,
Continuously tracking the location of the eyes of the user once the location of the eyes has been detected,
Measuring locations of glints by capturing the glints in the image sensors independently and calculating the coordinates of the glints in a POR calculation module,
Measuring a location of the pupil by illuminating the pupil and capturing a reflected pupil image,
Calculating an estimated POR value from the locations of glints in each eye and for each image sensor independently in a calculation module of the calculating unit,
Determining for each image sensor a confidence value (o) of the estimated POR, wherein captured parameters are compared to predetermined data ranges.

11. The method according to claim 10 , wherein a corrected POR value is determined by using the confidence value as a weight factor and performing a weighted average calculation of the estimated POR's from the light sensors.

12. The method according to claim 11 , wherein a corrected POR value is determined by analyzing the confidence value of each estimated POR, whereby only the estimated POR with the highest confidence value is considered for determining the corrected POR value.

13. The method according to claim 11 , wherein the confidence value (σ) is calculated from head pose correction W n hp , feature detection reliability W n fd and user distance W n ud , and wherein a separate confidence value σ n is calculated simultaneously for each estimated POR value from each image sensor and for each eye.

14. The method according to claim 11 , wherein a total confidence value is determined by a pre-estimation of the confidence value based on gaze data and a post estimation of the confidence value, whereby the post estimation determines data outliers in the estimated PORs.

15. The method according to claim 10 , wherein the locations of the eyes are only detected one time, preferably at the beginning of a session where after the eye positions are continuously tracked.

16. The method according to claim 10 , wherein at least four glints are captured and entered in the gaze tracking module as corners of a trapezium.

17. The method according to claim 10 , wherein the estimated POR value is calculated by a cross-ratio based module in which the image of the reflected image is compared to the geometry of the display.

Assignees

• Logitech Europe Sa
• Logitech Europe Sa

Inventors

• Arar Nuri Murat
• Thiran Jean-Philippe
• Theytaz Olivier

Classifications

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T7/004

  Physics · mapped topic

• G06T2207/30196

  Human being; Person · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06V40/193

  Preprocessing; Feature extraction · CPC title