Training an eye tracking model

Patent metadata

Field	Value
Publication number	US-11386290-B2
Application number	US-202016834153-A
Country	US
Kind code	B2
Filing date	Mar 30, 2020
Priority date	Mar 29, 2019
Publication date	Jul 12, 2022
Grant date	Jul 12, 2022

Abstract

A method for training an eye tracking model is disclosed, as well as a corresponding system and storage medium. The eye tracking model is adapted to predict eye tracking data based on sensor data from a first eye tracking sensor. The method comprises receiving sensor data obtained by the first eye tracking sensor at a time instance and receiving reference eye tracking data for the time instance generated by an eye tracking system comprising a second eye tracking sensor. The reference eye tracking data is generated by the eye tracking system based on sensor data obtained by the second eye tracking sensor at the time instance. The method comprises training the eye tracking model based on the sensor data obtained by the first eye tracking sensor at the time instance and the generated reference eye tracking data.

First claim

The invention claimed is:

1. A method for training an eye tracking model, wherein
the eye tracking model is adapted to predict eye tracking data based on sensor data from a first eye tracking sensor, wherein the first eye tracking sensor is arranged to monitor an eye, the method comprising:
receiving sensor data obtained by the first eye tracking sensor at a time instance;
receiving reference eye tracking data for said time instance generated by an eye tracking system comprising a second eye tracking sensor, arranged to monitor the eye, wherein the reference eye tracking data is generated by the eye tracking system based on sensor data obtained by the second eye tracking sensor at said time instance; and
training the eye tracking model based on the sensor data obtained by the first eye tracking sensor at said time instance and the generated reference eye tracking data.

2. The method of claim 1 , further comprising:
using the first eye tracking sensor to obtain sensor data at said time instance.

3. The method of claim 1 , further comprising:
using the eye tracking system to generate the reference eye tracking data for said time instance.

4. The method of claim 1 ,
wherein the eye tracking data predicted by the eye tracking model indicates a predicted gaze point of the eye, and wherein the generated reference eye tracking data indicates a reference gaze point of the eye; or
wherein the eye tracking data predicted by the eye tracking model indicates a predicted gaze ray of the eye, and wherein the generated reference eye tracking data indicates a reference gaze ray of the eye; or
wherein the eye tracking data predicted by the eye tracking model indicates a predicted position of the eye in space, and wherein the generated reference eye tracking data indicates a reference position of the eye in space.

5. The method of claim 1 , wherein training the eye tracking model comprises:
predicting eye tracking data for said time instance using the eye tracking model and the sensor data obtained by the first eye tracking sensor at said time instance;
applying an objective function to at least the eye tracking data predicted by the eye tracking model for said time instance and the generated reference eye tracking data; and
updating the eye tracking model.

6. The method of claim 5 , wherein applying the objective function to at least the eye tracking data predicted by the eye tracking model for said time instance and the generated reference eye tracking data comprises:
forming a distance between a predicted gaze point indicated by the predicted eye tracking data for said time instance and a reference gaze point indicated by the generated reference eye tracking data; or
forming a deviation between a predicted gaze ray indicated by the predicted eye tracking data for said time instance and a reference gaze ray indicated by the generated reference eye tracking data; or
forming a distance between a predicted eye position indicated by the predicted eye tracking data for said time instance and a reference eye position indicated by the generated reference eye tracking data.

7. The method of claim 1 , wherein at least one of the first eye tracking sensor and the second eye tracking sensor is an imaging device.

8. The method of claim 1 , wherein:
the eye tracking system comprises an illuminator which outputs light within a wavelength range for illuminating the eye, the second eye tracking sensor providing sensor data based on light within said wavelength range, the first eye tracking sensor being provided with a filter for suppressing light within said wavelength range.

9. The method of claim 1 , wherein training the eye tracking model comprises:
predicting eye tracking data for said time instance using the eye tracking model and the sensor data obtained by the first eye tracking sensor at said time instance; and
in response to a deviation between the eye tracking data predicted by the eye tracking model for said time instance and the generated reference eye tracking data exceeding a threshold, training the eye tracking model based on the eye tracking data predicted by the eye tracking model for said time instance and the generated reference eye tracking data.

10. The method of claim 1 , further comprising:
using the eye tracking system to detect a certain trigger action of the eye,

wherein the trigger action comprises one or more of
a fixation;
a saccade;
a smooth pursuit;

wherein the receiving of the sensor data obtained by the first eye tracking sensor at the time instance or the training of the eye tracking model is performed in response to detection of the certain trigger action of the eye.

11. The method of claim 1 , wherein the eye tracking model is one of several eye tracking models, the eye tracking models being associated with respective potential users, the method comprising:
detecting presence of a user;
selecting the eye tracking model associated with the user; and
training the selected eye tracking model based on the sensor data obtained by the first eye tracking sensor at said time instance and the generated reference eye tracking data.

12. The method of claim 1 , comprising:
receiving sensor data obtained by the first eye tracking sensor at a sequence of time instances;
receiving reference eye tracking data for the sequence of time instances generated by the eye tracking system, wherein the reference eye tracking data for the sequence of time instances is generated by the eye tracking system based on sensor data obtained by the second eye tracking sensor at said sequence of time instances; and
training the eye tracking model based on the sensor data obtained by the first eye tracking sensor for the sequence of time instances and the generated reference eye tracking data for the sequence of time instances, or storing the sensor data obtained by the first eye tracking sensor for the sequence of time instances and the generated reference eye tracking data for the sequence of time instances.

13. A system for training an eye tracking model, wherein
the eye tracking model is adapted to predict eye tracking data based on sensor data from a first eye tracking sensor, wherein the first eye tracking sensor is arranged to monitor an eye, the system comprising processing circuitry configured to:
receive sensor data obtained by the first eye tracking sensor at a time instance;
receive reference eye tracking data for said time instance generated by an eye tracking system comprising a second eye tracking sensor, arranged to monitor the eye, wherein the reference eye tracking data is generated by the eye tracking system based on sensor data obtained by the second eye tracking sensor at said time instance; and
train the eye tracking model based on the sensor data obtained by the first eye tracking sensor at said time instance and the generated reference eye tracking data.

14. The system of claim 13 , further comprising the first eye tracking sensor, the processing circuitry being further configured to:
use the first eye tracking sensor to obtain sensor data at said time instance.

15. The system of claim 13 , further comprising the eye tracking system, the processing circuitry being further configured to:
use the eye tracking system to generate the reference eye tracking data for said time instance.

16. The system of claim 13 , wherein at least one of the first eye tracking sensor and the second eye tracking sensor is an imaging device.

17. The system of claim 13 , wherein the eye tracking system comprises an illuminator which is config

Assignees

• Tobii Ab

Inventors

• Asplund Carl
• Barkman Patrik
• Dahl Anders
• Danielsson Oscar
• Martini Tommaso
• Nilsson Mårten

Classifications

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F18/214 Primary

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G01S7/4008

  of transmitters · CPC title

• G01S13/003

  Bistatic radar systems; Multistatic radar systems · CPC title

• G01S7/285

  Receivers · CPC title