Detecting relocation of a head-mounted device

Patent metadata

Field	Value
Publication number	US-11259010-B2
Application number	US-201916669914-A
Country	US
Kind code	B2
Filing date	Oct 31, 2019
Priority date	Oct 31, 2018
Publication date	Feb 22, 2022
Grant date	Feb 22, 2022

Abstract

Images of an eye are captured at respective time instances by a camera of a head-mounted device. For each time instance, a position of a center of corneal curvature is estimated using an image captured at that time instance, a position of a pupil center is estimated using an image captured at that time instance, and a line is determined through the estimated corneal curvature center position and the estimated pupil center position. A first estimated position of a center of the eye is computed based on the lines determined for time instances in a first time period. A second estimated position of the center of the eye is computed based on the lines determined for time instances in a second time period. Relocation of the head-mounted device relative to a user's head is detected based on the first and second estimated positions of the eye center.

First claim

The invention claimed is:

1. A method for detecting relocation of a head-mounted device relative to a user's head, the method comprising:
obtaining images of an eye captured at a sequence of time instances by a camera of the head-mounted device;
for each of the time instances:
estimating, using at least one image of the eye captured at that time instance, a position of a center of a corneal curvature of the eye,
estimating, using at least one image of the eye captured at that time instance, a position of a center of a pupil of the eye, and
determining a line through the estimated position of the center of the corneal curvature and the estimated position of the center of the pupil;

computing a first estimated position of a center of the eye based on the lines determined for time instances in a first time period, wherein the first time period comprises a first plurality of time instances;
computing a second estimated position of the center of the eye based on the lines determined for time instances in a second time period, wherein the second time period comprises a second plurality of time instances; and
subsequent to computing the first estimated position and the second estimated position, detecting, based on a comparison of the first and second estimated positions of the center of the eye, relocation of the head-mounted device relative to the user's head.

2. The method of claim 1 , wherein:
the first estimated position is computed based on an approximated intersection point between the lines determined for time instances in the first time period; and/or
the second estimated position is computed based on an approximated intersection point between the lines determined for time instances in the second time period.

3. The method of claim 1 , wherein computing the first estimated position of the center of the eye comprises:
forming a first set of distances between candidate positions and the lines determined for time instances in the first time period;
forming a first cost function based on the first set of distances; and
computing the first estimated position of the center of the eye as a candidate position minimizing the first cost function,

and/or wherein computing the second estimated position of the center of the eye comprises:
forming a second set of distances between candidate positions and the lines determined for time instances in the second time period;
forming a second cost function based on the second set of distances; and
computing the second estimated position of the center of the eye as a candidate position minimizing the second cost function.

4. The method of claim 1 , wherein the first and second estimated positions of the center of the eye are computed as positions in a predefined plane.

5. The method of claim 1 , wherein the first and second estimated positions of the center of the eye are computed as positions in a plane of an image captured by said camera.

6. The method of claim 4 , wherein the positions of the center of the corneal curvature and the positions of the center of the pupil are estimated in said plane, and wherein the lines are determined in said plane.

7. The method of claim 4 , wherein:
a) the positions of the center of the corneal curvature and the positions of the center of the pupil are estimated as positions in a three-dimensional space, the lines are determined in the three-dimensional space, and the first and second estimated positions of the center of the eye are computed based on projections of the lines in said plane; or
b) the positions of the center of the corneal curvature and the positions of the center of the pupil are estimated as positions in a three-dimensional space, the lines are determined in the three-dimensional space, and the first and second estimated positions of the center of the eye are computed in the three-dimensional space and are then projected down to said plane.

8. The method of claim 1 , wherein the positions of the center of the corneal curvature, the positions of the center of the pupil, and the positions of the center of the eye are estimated as positions in a three-dimensional space.

9. The method of claim 1 , wherein at least some of the images of the eye are captured while the eye is illuminated by an illuminator, and wherein, for a time instance, the position of the center of the corneal curvature is estimated based on a position of a reflection of the illuminator at a cornea of the eye in an image captured at that time instance.

10. The method of claim 1 , wherein at least some of the images of the eye are captured while the eye is illuminated by an illuminator, and wherein, for a time instance, the position of the center of the pupil is estimated based on a position of the center of the pupil in an image of the eye captured at that time instance and based on a reflection of the illuminator at a cornea of the eye in an image captured at that time instance.

11. The method of claim 1 , wherein detecting relocation of the head-mounted device relative to the user's head comprises:
detecting relocation of the head-mounted device in response to a deviation between the first and second estimated positions of the center of the eye exceeding a threshold.

12. The method of claim 1 , further comprising:
obtaining images of a second eye captured at the sequence of time instances by a camera of the head-mounted device;
for each of the time instances:
estimating, using at least one image of the second eye captured at that time instance, a position of a center of a corneal curvature of the second eye,
estimating, using at least one image of the second eye captured at that time instance, a position of a center of a pupil of the second eye, and
determining a line through the estimated position of the center of the corneal curvature of the second eye and the estimated position of the center of the pupil of the second eye;

computing a first estimated position of a center of the second eye based on the lines determined for the second eye for time instances in the first time period, wherein the first time period comprises a first plurality of time instances; and
computing a second estimated position of the center of the second eye based on lines determined for the second eye for time instances in the second time period, wherein the second time period comprises a second plurality of time instances,
wherein the detection of relocation of the head-mounted device relative to the user's head is also based on the first and second estimated positions of the center of the second eye.

13. The method of claim 12 , wherein detecting relocation of the head-mounted device relative to the user's head comprises:
forming a first distance between the first estimated position of the center of the first eye and the first estimated position of the center of the second eye;
forming a second distance between the second estimated position of the center of the first eye and the second estimated position of the center of the second eye; and
checking that a difference between the first and second distances is below a threshold.

14. The method of claim 1 , further comprising, in response to detecting relocation of the head-mounted device relative to the user's head:
providing signaling for calibration of the head-mounted device; or
providing signaling for prompting the user to reposition the head-mounted device.

15. A system for detecting relocation of a head-mounted device relative to a user's head, the system comprising processing circuitry configured to:
obtain images of an eye captured at a sequence of time instances by a camera of the

Assignees

• Tobii Ab

Inventors

• Wang Tiesheng
• Rana Pravin Kumar
• Wang Yimu
• Ryan Mark

Classifications

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G02B27/017

  Head mounted · CPC title