Method and an apparatus for determining a gaze point on a three-dimensional object

Patent metadata

Field	Value
Publication number	US-12423865-B2
Application number	US-202418581765-A
Country	US
Kind code	B2
Filing date	Feb 20, 2024
Priority date	Sep 17, 2012
Publication date	Sep 23, 2025
Grant date	Sep 23, 2025

Abstract

A system for determining the gaze endpoint of a subject, the system comprising: a eye tracking unit adapted to determine the gaze direction of one or more eyes of the subject; a head tracking unit adapted to determine the position comprising location and orientation of the eye tracker with respect to a reference coordinate system; a 3D Structure representation unit, that uses the 3D structure and position of objects of the scene in the reference coordinate system to provide a 3D structure representation of the scene; based on the gaze direction, the eye tracker position and the 3D structure representation, calculating the gaze endpoint on an object of the 3D structure representation of the scene or determining the object itself.

First claim

What is claimed is:

1. A method comprising:
at a computing device including an eye tracker:
measuring, by the eye tracker, gaze directions of two eyes of a subject in a 3D space;
determining, by the computing device, a gaze point of the subject in the 3D space based on the gaze directions; and
choosing, by the computing device, a 3D object in the 3D space as the 3D object the subject is gazing at based on the gaze point and a location in the 3D space of a representative 3D representation that represents the 3D object and has a 3D extension in the 3D space, wherein the gaze point is within a region of the 3D space surrounding and excluding the 3D object.

2. The method of claim 1 , wherein the gaze point is determined based on the vergence of the two eyes using convergence of the gaze directions.

3. The method of claim 1 , wherein a point lying closest to the gaze directions is chosen as the gaze point when the gaze directions do not intersect.

4. A non-transitory computer-readable medium having instructions encoded thereon which, when executed by one or more processors of a computing device with an eye tracker, cause the computing device to:
measure, by the eye tracker, gaze directions of two eyes of a subject in a 3D space;
determine, by the computing device, a gaze point of the subject in the 3D space based on the gaze directions; and
choose, by the computing device, a 3D object in the 3D space as the 3D object the subject is gazing at based on the gaze point and a location in the 3D space of a representative 3D representation that represents the 3D object and has an extension in the 3D space, wherein the gaze point is within a region of the 3D space surrounding and excluding the 3D object.

5. The non-transitory computer-readable medium of claim 4 , wherein the gaze point is determined based on the vergence of the two eyes using convergence of the gaze directions.

6. The non-transitory computer-readable medium of claim 4 , wherein a point lying closest to the gaze directions is chosen as the gaze endpoint when the gaze directions do not intersect.

7. A computing device with an eye tracker comprising:
a memory to store a representative 3D representation that represents a 3D object in a 3D space and has a 3D extension in the 3D space; and
one or more processors to:
measure, by the eye tracker, gaze directions of two eyes of a subject in the 3D space;
determine, by the computing device, a gaze point of the subject in the 3D space based on the gaze directions; and
choose, by the computing device, the 3D object as the 3D object the subject is gazing at based on the gaze point and a location in the 3D space of the representative 3D representation, wherein the gaze point is within a region of the 3D space surrounding and excluding the 3D object.

8. The computing device of claim 7 , wherein the gaze point is determined based on the vergence of the two eyes using convergence of the gaze directions.

9. The computing device of claim 7 , wherein a point lying closest to the gaze directions is chosen as the gaze endpoint when the gaze directions do not intersect.

10. The method of claim 1 , wherein a 3D shape of the representative 3D representation is different than a 3D shape of the 3D object.

11. The method of claim 10 , wherein the 3D shape of the representative 3D representation is a sphere.

12. The method of claim 10 , wherein the 3D shape of the representative 3D representation is space tessellating.

13. The method of claim 1 , wherein the gaze point is within the representative 3D representation.

14. The method of claim 1 , wherein the gaze point is a positive distance from the location in the 3D space of the representative 3D representation.

15. The method of claim 14 , wherein choosing the 3D object in the 3D space is based on the positive distance between the gaze point and the location in the 3D space of the representative 3D representation.

16. The method of claim 15 , wherein choosing the 3D object in the 3D space includes determining that the positive distance between the gaze point and the location in the 3D space of the representative 3D representation is the shortest distance between the gaze point and any location of any representative 3D representation of any 3D object in the 3D space.

17. The method of claim 1 , wherein choosing the 3D object in the 3D space is based on a probability that the 3D object in the 3D space is the 3D object the subject is gazing at.

18. The non-transitory computer-readable medium of claim 4 , wherein a 3D shape of the representative 3D representation is different than a 3D shape of the 3D object.

19. The non-transitory computer-readable medium of claim 4 , wherein the gaze point is within the representative 3D representation.

20. The non-transitory computer-readable medium of claim 4 , wherein the gaze point is a positive distance from the location in the 3D space of the representative 3D representation.

Assignees

• Apple Inc
• Aplle Inc

Inventors

• Hoffmann Jan
• Sengelaub Tom
• Williams Denis

Classifications

• G06T2207/30201

  Face · CPC title

• G06T2207/20076

  Probabilistic image processing · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2200/04

  involving 3D image data · CPC title

• G06V40/19 Primary

  Sensors therefor · CPC title