System and method for determining reference gaze data

Patent metadata

Field	Value
Publication number	US-12527474-B2
Application number	US-202217699588-A
Country	US
Kind code	B2
Filing date	Mar 21, 2022
Priority date	Mar 30, 2021
Publication date	Jan 20, 2026
Grant date	Jan 20, 2026

Abstract

The invention is related to an eye tracking system for determining reference gaze data of a user in a scene exposing a pupil of the user. The eye tracking system comprising processing circuitry configured to obtain a first eye image comprising the pupil of the user, the first eye image being captured during a first time period; determine, based on the first eye image, a first pupil size; obtain a second eye image comprising the pupil of the user, the second eye image being captured during a second time period; determine, based on the second eye image, a second pupil size; obtain scene information of the scene exposing the pupil of the user, the scene information comprising at least the first luminance level, the second luminance level and spatial information of the second region during the second time period; determine a pupil size change between the first pupil size and the second pupil size, the pupil size change indicating that the user is looking at the second region; determine reference gaze data of the user during the second time period, if the pupil size change is larger than a pupil size change threshold. The invention further relates to a head-mounted device, a method, a computer program and a carrier.

First claim

The invention claimed is:

1 . An eye tracking system for determining reference gaze data of a user in a scene exposing a pupil of the user, the scene comprising at least a first region having a first luminance level and a second region having a second luminance level differing from the first luminance level by more than a luminance level threshold, the eye tracking system comprising processing circuitry configured to:
obtain a first eye image comprising the pupil of the user, the first eye image being captured during a first time period; determine, based on the first eye image, a first pupil size; obtain a second eye image comprising the pupil of the user, the second eye image being captured during a second time period; determine, based on the second eye image, a second pupil size; obtain scene information of the scene exposing the pupil of the user, the scene information comprising at least the first luminance level, the second luminance level and spatial information indicating a position of the second region during the second time period; determine a pupil size change between the first pupil size and the second pupil size, the pupil size change indicating that the user is looking at the second region; determine reference gaze data of the user during the second time period, if the pupil size change is larger than a pupil size change threshold, wherein the reference gaze data comprises at least one of a gaze point or a gaze ray indicating where the user is looking; and link the reference gaze data to the second region based on the spatial information of the second region to confirm that the user is looking at the second region.

2 . The eye tracking system according to claim 1 , the processing circuitry further being configured to:
obtain a position of the pupil during the second time period; and wherein the determining of reference gaze data of the user is at least partly based on the obtained position of the pupil.

3 . The eye tracking system according to claim 2 , wherein the obtaining of the position of the pupil is at least partly based on the position of the pupil in the second eye image.

4 . The eye tracking system according to claim 1 , wherein the determining of reference gaze data of the user is at least partly based on the obtained scene information.

5 . The eye tracking system according to claim 1 , the processing circuitry further being configured to:
determine a distance difference between the reference gaze data and the spatial information of the second region; and send an error signal to the user if the distance difference is above an error threshold.

6 . The eye tracking system according to claim 1 , the processing circuitry further being configured to:
update a gaze calibration setting of the user, based on the reference gaze data and the scene information.

7 . The eye tracking system according to claim 6 , wherein the reference gaze data comprises a gaze convergence depth, wherein the spatial information of the second region comprises depth information and wherein the updating of the gaze calibration setting of the user further comprises:
updating the gaze convergence depth of the user based on the depth information.

8 . The eye tracking system according to claim 1 , the processing circuitry further being configured to:
obtain a first total light level exposing the pupil of the user during the first time period; obtain a second total light level exposing the pupil of the user during the second time period; determine a total light level change between the first total light level and the second total light level; and only determine reference gaze data if the total light level change is smaller than a total light level change threshold.

9 . The eye tracking system according to claim 1 , the processing circuitry further being configured to:
obtain a first scene image exposing the pupil of the user during the first time period; obtain a second scene image exposing the pupil of the user during the second time period; determine a scene change between the first scene image and the second scene image; and only determine reference gaze data of the user if the scene change is smaller than a scene change threshold.

10 . The eye tracking system according to claim 1 , the processing circuitry being further configured to:
determine, based on the first eye image, a first pupil position; determine, based on the second eye image, a second pupil position; determine a pupil position change between the first pupil position and the second pupil position; and only determine reference gaze data of the user if the pupil position change is larger than a pupil position change threshold.

11 . The eye tracking system according to claim 1 , wherein the reference gaze data comprises at least one of a gaze point and a gaze ray.

12 . The eye tracking system according to claim 1 , the eye tracking system further comprising a display arranged to display at least a part of the scene exposing the pupil of the user.

13 . The eye tracking system according to claim 1 , the eye tracking system further comprising an outward facing scene sensor arranged to obtain at least a part of the scene exposing the pupil of the user.

14 . The eye tracking system according to claim 1 , the eye tracking system further comprising an eye image sensor arranged to capture the first eye image and the second eye image.

15 . The eye tracking system according to claim 14 , the eye tracking system further comprising at least one illuminator, wherein each illuminator is arranged to cause a glint in the eye of the user, the glint being detectable in the second eye image, wherein the determining of reference gaze data of the user is at least partly based on a position of the at least one glint in relation to the position of the pupil.

16 . A head-mounted device comprising an eye-tracking system for determining reference gaze data of a user in a scene exposing a pupil of the user, the scene comprising at least a first region having a first luminance level and a second region having a second luminance level differing from the first luminance level by more than a luminance level threshold, the eye tracking system comprising processing circuitry configured to:
obtain a first eye image comprising the pupil of the user, the first eye image being captured during a first time period; determine, based on the first eye image, a first pupil size; obtain a second eye image comprising the pupil of the user, the second eye image being captured during a second time period; determine, based on the second eye image, a second pupil size; obtain scene information of the scene exposing the pupil of the user, the scene information comprising at least the first luminance level, the second luminance level and spatial information indicating a position of the second region during the second time period; determine a pupil size change between the first pupil size and the second pupil size, the pupil size change indicating that the user is looking at the second region; determine reference gaze data of the user during the second time period, if the pupil size change is larger than a pupil size change threshold, wherein the reference gaze data comprises at least one of a gaze point or a gaze ray indicating where the user is looking; and link the reference gaze data to the second region based on the spatial information of the second region to confirm that the user is looking at the second region.

17 . A method, performed by an eye tracking system, for determining reference gaze

Assignees

• Tobii Ab

Inventors

• Andersson Henrik
• Akkil Deepak

Classifications

• A61B3/14

  Arrangements specially adapted for eye photography · CPC title

• A61B3/0025

  characterised by electronic signal processing, e.g. eye models · CPC title

• A61B3/0008

  provided with illuminating means (A61B3/117, A61B3/14 take precedence) · CPC title

• A61B5/163

  by tracking eye movement, gaze, or pupil change · CPC title

• A61B3/112 Primary

  for measuring diameter of pupils · CPC title