Polarized gaze tracking

Patent metadata

Field	Value
Publication number	US-9330302-B2
Application number	US-201414191305-A
Country	US
Kind code	B2
Filing date	Feb 26, 2014
Priority date	Feb 26, 2014
Publication date	May 3, 2016
Grant date	May 3, 2016

Abstract

Embodiments that relate to determining gaze locations are disclosed. In one embodiment a method includes shining light along an outbound light path to the eyes of the user wearing glasses. Upon detecting the glasses, the light is dynamically polarized in a polarization pattern that switches between a random polarization phase and a single polarization phase, wherein the random polarization phase includes a first polarization along an outbound light path and a second polarization orthogonal to the first polarization along a reflected light path. The single polarization phase has a single polarization. During the random polarization phases, glares reflected from the glasses are filtered out and pupil images are captured. Glint images are captured during the single polarization phase. Based on pupil characteristics and glint characteristics, gaze locations are repeatedly detected.

First claim

The invention claimed is:

1. A method for determining gaze locations of an eye of a user, the method comprising:
shining light along an outbound light path from a light source to the eyes of the user wearing glasses;
detecting that the user is wearing glasses using an image captured by an image capture device;
upon detecting that the user is wearing glasses, dynamically polarizing the light in a polarization pattern that repeatedly switches between a random polarization phase and a single polarization phase at a rate of at least 60 Hz, wherein the random polarization phase includes a first polarization of the light along the outbound light path intermediate the light source and the glasses of the user and a second polarization orthogonal to the first polarization along a reflected light path intermediate the glasses and the image capture device, and the single polarization phase having a single polarization along one or more of the outbound light path and the reflected light path;
during the random polarization phases, filtering out glares reflected from the glasses that would otherwise occlude a pupil of the eye;
during the random polarization phases when the glares are filtered out, capturing pupil images at a rate of 30 Hz or higher;
during the single polarization phases, capturing glint images at a rate of 30 Hz or higher; and
repeatedly detecting the gaze locations at a rate of at least 30 Hz based on pupil characteristics identified in the pupil images and glint characteristics identified in the glint images captured proximate in time to the pupil images.

2. The method of claim 1 , wherein detecting that the user is wearing glasses further comprises determining that one or more of the glares is located in the vicinity of the eye of the user in the captured image.

3. The method of claim 1 , wherein dynamically polarizing the light in a polarization pattern further comprises repeatedly switching between the random polarization phase and the single polarization phase at a rate of 120 Hz.

4. The method of claim 1 , wherein the pupil images and the glint images are captured at a rate of 60 Hz., and the gaze locations are repeatedly detected at a rate of 60 Hz.

5. The method of claim 1 , further comprising:
detecting that the user is not wearing glasses;
upon detecting that the user is not wearing glasses, refraining from dynamically polarizing the light in the polarization pattern that repeatedly switches between the random polarization phase and the single polarization phase; and
capturing the pupil images and the glint images during the single polarization phases at a rate of 30 Hz or higher.

6. The method of claim 1 , wherein the single polarization of the single polarization phases further comprises applying the second polarization on the outbound light path.

7. The method of claim 1 , wherein dynamically polarizing the light comprises alternating a switchable polarizing filter between the first polarization and the second polarization orthogonal to the first polarization.

8. The method of claim 1 , wherein the light source is a first light source, and a second light source emits unpolarized light to the eyes of the user wearing glasses, wherein the unpolarized light is used to capture the glint images.

9. The method of claim 1 , wherein the image capture device is a first image capture device that captures the pupil images, and the method further comprises using a second image capture device that captures the glint images.

10. A gaze tracking system for determining gaze locations of an eye of a user, the gaze tracking system comprising:
a light source for shining light along an outbound light path to the eyes of the user wearing glasses;
a polarizing filter configured to dynamically polarize the light;
an image capture device configured to capture images of the light reflected and scattered from the eye of the user and the glasses;
a computing device operatively connected to at least the light source and the image capture device;
a dynamic polarization module executed by a processor of the computing device, the dynamic polarization module configured to:
detect that the user is wearing glasses using an image captured by the image capture device;
upon detecting that the user is wearing glasses, dynamically polarize the light in a polarization pattern that repeatedly switches between a random polarization phase and a single polarization phase at a rate of at least 60 Hz, wherein the random polarization phase includes a first polarization of the light along the outbound light path intermediate the light source and the glasses of the user and a second polarization orthogonal to the first polarization along a reflected light path intermediate the glasses and the image capture device, and the single polarization phase having a single polarization along one or more of the outbound light path and the reflected light path;
during the random polarization phases, filter out glares reflected from the glasses that would otherwise occlude a pupil of the eye;
during the random polarization phases when the glares are filtered out, capture pupil images with the image capture device at a rate of 30 Hz or higher; and
during the single polarization phases, capture glint images with the image capture device at a rate of 30 Hz or higher; and

a gaze tracking module configured to repeatedly detect the gaze locations at a rate of at least 30 Hz based on pupil characteristics identified in the pupil images and glint characteristics identified in the glint images captured proximate in time to the pupil images.

11. The gaze tracking system of claim 10 , wherein the dynamic polarization module is further configured to detect that the user is wearing glasses by determining that one or more of the glares is located in the vicinity of the eye of the user in the captured image.

12. The gaze tracking system of claim 10 , wherein the dynamic polarization module is further configured to dynamically polarize the light in a polarization pattern by repeatedly switching between the random polarization phase and the single polarization phase at a rate of 120 Hz.

13. The gaze tracking system of claim 10 , wherein the dynamic polarization module is further configured to capture the pupil images and the glint images at a rate of 60 Hz., and detect the gaze locations at a rate of 60 Hz.

14. The gaze tracking system of claim 10 , wherein the dynamic polarization module is further configured to:
detect that the user is not wearing glasses;
upon detecting that the user is not wearing glasses, refrain from dynamically polarizing the light in the polarization pattern that repeatedly switches between the random polarization phase and the single polarization phase; and
capture the pupil images and the glint images during the single polarization phases at a rate of 30 Hz or higher.

15. The gaze tracking system of claim 10 , wherein the single polarization of the single polarization phases comprises applying the second polarization on the outbound light path.

16. The gaze tracking system of claim 10 , wherein the dynamic polarization module is further configured to dynamically polarize the light by alternating a switchable polarizing filter between the first polarization and the second polarization orthogonal to the first polarization.

17. The gaze tracking system of claim 10 , wherein the light source is a first light source, wherein the single polarization phase has the single polarization along the reflected light path, and further comprising a second light source that emits unpolarized light to the

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Thukral Vaibhav
• Sinha Sudipta
• Pradeep Vivek
• Large Timothy Andrew
• Keam Nigel Stuart
• Nister David

Classifications

• A61B3/113 Primary

  for determining or recording eye movement · CPC title

• G06V40/20 Primary

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title

• H04N23/74

  by influencing the scene brightness using illuminating means · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• G06V10/141

  Control of illumination · CPC title