Gaze tracking via eye gaze model

Patent metadata

Field	Value
Publication number	US-9864430-B2
Application number	US-201514593955-A
Country	US
Kind code	B2
Filing date	Jan 9, 2015
Priority date	Jan 9, 2015
Publication date	Jan 9, 2018
Grant date	Jan 9, 2018

Abstract

Examples are disclosed herein that are related to gaze tracking via image data. One example provides, on a gaze tracking system comprising an image sensor, a method of determining a gaze direction, the method comprising acquiring image data via the image sensor, detecting in the image data facial features of a human subject, determining an eye rotation center based upon the facial features using a calibrated face model, determining an estimated position of a center of a lens of an eye from the image data, determining an optical axis based upon the eye rotation center and the estimated position of the center of the lens, determining a visual axis by applying an adjustment to the optical axis, determining the gaze direction based upon the visual axis, and providing an output based upon the gaze direction.

First claim

The invention claimed is:

1. On a gaze tracking system comprising an image sensor, a method of determining a gaze direction, the method comprising:
acquiring image data via the image sensor;
detecting in the image data facial features of a human subject;
determining an eye rotation center based upon the facial features using a calibrated face model previously calibrated to a face of the human subject, the calibrated face model relating an offset of the eye rotation center to the facial features;
determining an estimated position of a center of a lens of an eye from the image data;
determining an optical axis based upon the eye rotation center and the estimated position of the center of the lens, the optical axis extending through the eye rotation center and the estimated position of the center of the lens;
determining a visual axis by applying an adjustment to the optical axis;
determining the gaze direction based upon the visual axis; and
providing an output based upon the gaze direction determined from applying the calibrated face model.

2. The method of claim 1 , wherein the image sensor comprises a 2D visible light image sensor, and wherein the image data comprises visible image data.

3. The method of claim 2 , wherein detecting the facial features comprises locating 2D positions of the facial features in the visible image data, and further comprising determining 3D positions of the facial features from the 2D positions.

4. The method of claim 2 , wherein the estimated position of the center of the lens comprises a position of an iris center.

5. The method of claim 1 , wherein the image sensor comprises an infrared camera, and wherein the image data comprises infrared image data, and wherein the estimated position of the center of the lens comprises a position of a pupil center.

6. The method of claim 1 , wherein the image data comprises two-dimensional image data and depth image data.

7. The method of claim 1 , further comprising detecting in the image data facial features of a plurality of human subjects, and obtaining a calibrated face model for each of the plurality of human subjects.

8. The method of claim 1 , further comprising estimating a head pose of the human subject based on the facial features.

9. The method of claim 1 , wherein determining the eye rotation center based upon the facial features comprises determining a calibrated offset between the eye rotation center and one or more of the facial features.

10. The method of claim 1 , wherein applying the adjustment to the optical axis comprises applying a calibrated offset to the optical axis.

11. A gaze tracking system comprising:
an image sensor;
a logic subsystem; and
a storage subsystem comprising instructions executable by the logic subsystem to:
acquire image data,
detect in the image data facial features of a human subject,
determine an eye rotation center based upon the facial features using a calibrated face model previously calibrated to a face of the human subject, the calibrated face model relating an offset of the eye rotation center to the facial features,
determine an estimated position of a center of a lens of an eye from the image data,
determine an optical axis based upon the eye rotation center and the estimated position of the center of the lens,
determine a visual axis by applying an adjustment to the optical axis,
determine a gaze direction based upon the visual axis, and
provide an output based upon the gaze direction determined from applying the calibrated face model.

12. The gaze tracking system of claim 11 , further comprising instructions executable by the logic subsystem to determine the eye rotation center based upon 2D positions of the facial features using the calibrated face model.

13. The gaze tracking system of claim 11 , further comprising an infrared image sensor.

14. The gaze tracking system of claim 11 , further comprising a visible light image sensor.

15. The gaze tracking system of claim 11 , further comprising instructions executable by the logic subsystem to detect in the image data facial features of a plurality of human subjects, and to obtain a calibrated face model for each of the plurality of human subjects.

16. The gaze tracking system of claim 11 , further comprising instructions executable by the logic subsystem to estimate a head pose based on the facial features.

17. The gaze tracking system of claim 11 , further comprising instructions executable by the logic subsystem to determine a calibrated offset between the eye rotation center and one or more of the facial features.

18. The gaze tracking system of claim 11 , further comprising instructions executable by the logic subsystem to determine the estimated position of the center of the lens by locating a center of an ellipse fitted to an iris of the eye in the image data.

19. A gaze tracking system comprising:
a visible light image sensor and a depth image sensor configured to acquire image data;
a logic subsystem; and
a storage subsystem comprising instructions executable by the logic subsystem to:
detect in the image data facial features of a human subject,
determine an eye rotation center based upon the facial features using a calibrated face model previously calibrated to a face of the human subject, the calibrated face model relating an offset of the eye rotation center to the facial features,
determine an estimated position of a center of a lens of an eye from the image data,
determine an optical axis based upon the eye rotation center and the estimated position of the center of the lens,
determine a visual axis by applying an adjustment to the optical axis,
determine a gaze direction based upon the visual axis, and
provide an output based upon the gaze direction determined from applying the calibrated face model.

20. The gaze tracking system of claim 19 , further comprising an infrared image sensor configured to acquire infrared image data.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Wu Dijia
• Conrad Michael J
• Burrell Tim
• Miao Xu
• Liu Zicheng
• Cai Qin
• Zhang Zhengyou

Classifications

• G06K9/0061

  Physics · mapped topic

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F3/038

  Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry · CPC title

• G06V40/193

  Preprocessing; Feature extraction · CPC title