Eye tracking using video information and electrooculography information

Patent metadata

Field	Value
Publication number	US-10757328-B2
Application number	US-201615390410-A
Country	US
Kind code	B2
Filing date	Dec 23, 2016
Priority date	Dec 23, 2016
Publication date	Aug 25, 2020
Grant date	Aug 25, 2020

Abstract

Disclosed are an apparatus and a method of low-latency, low-power eye tracking. In some embodiments, the eye tracking method operates a first sensor having a first level of power consumption that tracks positions of an eye of a user. In response to detection that the eye does not change position for a time period, the method stops operation of the first sensor and instead operates a second sensor that detects a change of position of the eye. The second sensor has a level of power consumption lower than the level of power consumption of the first sensor. Once the eye position changes, the second sensor resumes operation.

First claim

What is claimed is:

1. A method of eye tracking, comprising:
operating a video oculography (VOG) sensor to track a position of an eye of a user;
stopping operation of the VOG sensor in response to a first detection, based on VOG signals from the VOG sensor, that the eye does not change position for a time period;
operating an electrooculography (EOG) sensor to track the position of the eye responsive to stopping the operation of the VOG sensor, wherein the EOG sensor has a lower power consumption than the VOG sensor; and
resuming operation of the VOG sensor in response to a second detection, based on EOG signals from the EOG sensor, of a change of the position of the eye.

2. The method of claim 1 , wherein the VOG sensor has a first signal processing latency that is higher than a second signal processing latency of the EOG sensor.

3. The method of claim 1 , wherein the VOG sensor has a first eye-tracking accuracy that is greater than a second eye-tracking accuracy of the EOG sensor.

4. The method of claim 1 , wherein the VOG sensor comprises a camera that captures images of the eye of the user.

5. The method of claim 1 , wherein the EOG sensor comprises a low-power camera.

6. The method of claim 1 , further comprising:
estimating two eye positions based on the EOG signals of the EOG sensor at two frames and calculating a difference between the two eye positions; and
recognizing the change of the position of the eye when the difference between the two eye positions exceeds a threshold.

7. The method of claim 1 , further comprising:
continuing operation of the EOG sensor when the operation of the EOG sensor resumes.

8. An electronic device for eye tracking, comprising:
a viedo oculography (VOG) sensor to track a position of an eye of a user;
an electrooculography (EOG) sensor to track the position of the eye of the user, wherein the EOG sensor has a lower power consumption than the VOG sensor; and
a processor configured to execute a process including:
detecting, based on VOG signals from the VOG sensor, that the eye does not change position for a time period;
stopping operation of the VOG senror in response to detecting that the eye does not change position for the time period;
triggering the EOG sensor to track the position of the eye responsive to stopping the operation of the VOG sensor; and
resuming operation of the VOG sensor in response to a detection, based on EOG signals of the EOG sensor, of a change of the position of the eye.

9. The electronic device of claim 8 , further comprising:
another VOG sensor that tracks positions of another eye of the user; and
another EOG sensor that tracks the positions of the another eye of the user.

10. The electronic device of claim 8 , further comprising:
a light source that generates light that is reflected by the eye and captured by the VOG sensor or the EOG sensor.

11. The electronic device of claim 8 , further comprising:
a control circuit that controls an operation of the VOG sensor based on instructions from the processor.

12. The electronic device of claim 8 , wherein the processor is further configured to:
estimate two eye positions based on the EOG signals of the EOG sensor at two frames;
calculate a difference between the two eye positions; and
recognize the change of the position of the eye when the difference between the two eye positions exceeds a threshold.

13. The electronic device of claim 8 , wherein the VOG sensor has a first signal processing latency that is higher than a second signal processing latency of the EOG sensor, or the VOG sensor has a first eye-tracking accuracy that is better than a second eye-tracking accuracy of the EOG sensor.

14. The electronic device of claim 8 , where the VOG sensor is a camera, and the EOG sensor comprises a low-power camera.

15. A method of eye tracking, comprising:
operating a video oculography (VOG) sensor to track positions of an eye of a user at a first frame rate;
determining the positions of the eye at keyframes based on VOG signals collected from the VOG sensor at the first frame rate, wherein the VOG sensor stops operating based on the eye remaining in a fixed position for a time period;
operating am electrooculography (EOG) sensor to track the positions of the eye of the user at a second frame rate upon stopping an operation of the VOG sensor, wherein the EOG sensor has a lower power consumption than the VOG sensor, wherein the first frame rate is lower than the second frame rate;
determining the positions of the eye at regular frames based on EOG signals collected from the EOG sensor at the second frame rate;
determining a fixation point at a user interface based on a current position from the fixed position of the eye being tracked;
conducting a foveated rendering of the user interface based on the fixation point; and
resuming operation of the VOG sensor in response to a detection, based on the EOG signals from the EOG sensor, of a change of a position of the eye.

16. The method of claim 15 , further comprising:
generating a bias between an eye position determined based on the VOG signals collected from the VOG sensor at a keyframe and another eye position determined based on the EOG signals collected from the EOG sensor at a regular frame; and
calibrating multiple eye positions determined based on the EOG signals from the EOG sensor by subtracting the bias from the multiple eye positions.

17. The method of claim 15 , wherein the VOG sensor has a first signal processing latency that is higher than a second signal processing latency of the EOG sensor.

18. The method of claim 15 , wherein the VOG sensor has a first eye-tracking accuracy that is better than a second eye-tracking accuracy of the EOG sensor.

19. The method of claim 15 , wherein the EOG sensor comprises a low-power camera sensor.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Thukral Vaibhav
• Aholt Chris
• Mei Christopher Maurice
• Chau Bill
• Bach Nguyen
• Cherkashin Lev
• Kim Jaeyoun

Classifications

• H04N23/65 Primary

  Control of camera operation in relation to power supply · CPC title

• G06F1/163 Primary

  Wearable computers, e.g. on a belt · CPC title

• G06V40/19

  Sensors therefor · CPC title

• G06F1/1686

  the I/O peripheral being an integrated camera · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title