Eye gaze tracking using binocular fixation constraints

Patent metadata

Field	Value
Publication number	US-2017102768-A1
Application number	US-201615389333-A
Country	US
Kind code	A1
Filing date	Dec 22, 2016
Priority date	Mar 26, 2014
Publication date	Apr 13, 2017
Grant date	—

Abstract

The subject disclosure is generally directed towards eye gaze detection based upon both eyes being fixated on the same gaze location. In one aspect, an image including a subject's left and right eyes is captured, from which left and right glint and pupil center information are extracted. The left and right glint data and the left and right pupil data are used with left and right bias correction matrices to jointly determine a gaze location.

First claim

What is claimed is:

1 . A method comprising:
capturing an image that includes a left eye of a subject and a right eye of the subject; identifying, from the image, a plurality of glints from the left eye and a plurality of glints from the right eye as left glint data and right glint data, respectively; determining, from the image, pupil center information of the left eye and pupil center information of the right eye as left pupil data and right pupil data, respectively; combining the left glint data, left pupil data, right glint data, and right pupil data to determine a gaze location of the subject.

2 . The method of claim 1 , further comprising using the left glint data to compute a left homography matrix and the right glint data to compute a right homography matrix.

3 . The method of claim 2 , further comprising multiplying the left homography matrix with a left bias correction homography matrix and multiplying the right homography matrix with a right bias correction homography matrix.

4 . The method of claim 3 , further comprising, computing the left and right bias correction homography matrices comprising obtaining tuples of left eye glint data, and tuples of right eye glint data at a plurality of gaze calibration positions.

5 . The method of claim 4 , wherein computing the left and right bias correction homography matrices further comprises using the left eye glint data and the right eye glint data obtained at the plurality of gaze calibration positions to compute additional left homography matrices and additional right homography matrices at respective gaze calibration positions.

6 . The method of claim 5 , further comprising, computing a left screen-mapped pupil location and a right screen-mapped pupil location at the respective gaze calibration positions.

7 . The method of claim 6 , wherein computing the left and right bias correction homography matrices comprises:
computing the left bias correction homography matrix by determining a first distance between an actual gaze position and the left screen-mapped pupil location at the respective gaze calibration positions; and computing the right bias correction homography matrix by determining a second distance between the actual gaze position and the right screen-mapped pupil location at the respective gaze calibration positions.

8 . A system comprising:
a camera, the camera configured to capture a current image containing a first and second plurality of glints, a pupil center of the left eye, and a pupil center of the right eye; and a binocular computation corrector configured to:
compute a left homography matrix based upon the first plurality of glints, and the pupil center of the left eye;
compute a right homography matrix based upon the second plurality of glints, and the pupil center of the right eye; and
combine the left homography matrix with the right homography matrix to determine a gaze location of the subject.

9 . The system of claim 8 , wherein the binocular computation corrector is further configured to use a left bias correction homography matrix and a right bias correction homography matrix.

10 . The system of claim 9 , wherein the binocular computation corrector is further configured to perform a calibration process configured to obtain the left and right bias correction homography matrices, comprising:
obtaining tuples of left eye glint data at a plurality of gaze calibration positions; and obtaining tuples of right eye glint data at the plurality of gaze calibration positions; and obtaining tuples of left eye glint data, and tuples of right eye glint data at a plurality of gaze calibration positions.

11 . The system of claim 10 , wherein the calibration process is further configured to compute the left and right bias correction homography matrices using the left eye glint data and the right eye glint data obtained at the plurality of gaze calibration positions to compute additional left homography matrices and additional right homography matrices at respective gaze calibration positions.

12 . The system of claim 11 , wherein the calibration process is further configured to compute a left screen-mapped pupil location and a right screen-mapped pupil location at the respective gaze calibration positions.

13 . The system of claim 12 , wherein the calibration process is further configured to:
determine a first distance between an actual gaze position and the left screen-mapped pupil location at the respective gaze calibration positions to compute the left bias correction homography matrix; and determine a second distance between the actual gaze position and the right screen-mapped pupil location at the respective gaze calibration positions to compute the right bias correction homography matrix.

14 . The system of claim 8 , wherein capturing a current image comprises capturing a plurality of images for a plurality of gaze calibration positions, a respective one of the plurality of images corresponding to a respective one of the plurality of gaze calibration positions.

15 . One or more computer storage devices having executable instructions, which upon execution perform operations, comprising:
capturing an image that includes a left eye of a subject and a right eye of the subject; identifying, from the image, a plurality of glints from the left eye and a plurality of glints from the right eye as left glint data and right glint data, respectively; determining, from the image, pupil center information of the left eye and pupil center information of the right eye as left pupil data and right pupil data, respectively; combining the left glint data, left pupil data, right glint data, and right pupil data to determine a gaze location of the subject.

16 . The one or more computer storage devices of claim 15 , wherein capturing an image comprises capturing a plurality of images for a plurality of gaze calibration positions, a respective one of the plurality of images corresponding to a respective one of the plurality of gaze calibration positions.

17 . The one or more computer storage devices of claim 15 , having further executable instructions comprising using the left glint data to compute a left homography matrix and the right glint data to compute a right homography matrix.

18 . The one or more computer storage devices of claim 17 , further comprising multiplying the left homography matrix with the left bias correction matrix and multiplying the right homography matrix with the right bias correction matrix.

19 . The one or more computer storage devices of claim 17 , further comprising computing a left screen-mapped pupil location and a right screen-mapped pupil location.

20 . The one or more computer storage devices of claim 19 , further comprising determining a distance between an actual gaze position and the left screen-mapped pupil location to compute the left bias correction matrix, and determining a distance between the actual gaze position and the right screen-mapped pupil location to compute the right bias correction matrix.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Zhang Zhengyou
• Cai Qin

Classifications

• G06T7/85

  Stereo camera calibration · CPC title

• G06T2207/10012

  Stereo images · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06T7/0002

  Inspection of images, e.g. flaw detection · CPC title