Optical sightline tracking for a wearable system

Patent metadata

Field	Value
Publication number	US-12373030-B2
Application number	US-202217988707-A
Country	US
Kind code	B2
Filing date	Nov 16, 2022
Priority date	Nov 16, 2022
Publication date	Jul 29, 2025
Grant date	Jul 29, 2025

Abstract

Techniques are described for optical sightline tracking in a wearable system. Embodiments include a wearable structure that includes at least one integrated illuminator and at least one integrated imaging array. Each illuminator and imaging array is disposed in the periphery of an eye of a user wearing the structure. From the peripheral viewpoint, embodiments measure a three-dimensional origin point of one or both eyes and can compute angular coordinate measurements for one or both eyes based on the imaging to indicate the direction in which the user's eyes are looking (a sightline). Embodiments can track changes in the sightline to support various applications, such as positioning of display of virtual objects in virtual or augmented reality environments. Some embodiments can also use the obtained imagery for biometric verification and/or identification, detection of changes in pupillary response, etc.

First claim

What is claimed is:

1. A system for optical sightline tracking, the system comprising:
a wearable structure configured to be worn on a user's head;
a first illuminator integrated with the wearable structure at a first temporal side of a first eye of the user to project illumination across a first cornea of the first eye;
a first imaging array integrated with the wearable structure at the first temporal side to generate first imagery of the first eye; and
a control processor to:
compute a first three-dimensional (3D) origin point of the first eye based at least on detecting a first vertex of curvature of a first corneal surface of the first eye in the first imagery;
compute a first angular coordinate measurement of the first eye based on detecting a first set of pupil dimensions in the first imagery and computing an offset between the first set of pupil dimensions and a pupil dimensional reference; and
compute a sightline based on the first 3D origin point and the first angular coordinate measurement.

2. The system of claim 1 , wherein the control processor is to compute the first angular coordinate measurement of the first eye by:
detecting a first pupil center location in the first imagery and computing the offset as between the first pupil center location and the first vertex of curvature.

3. The system of claim 1 , wherein the control processor is to compute the first angular coordinate measurement of the first eye by:
detecting a roundness of the first pupil and computing the offset as between the roundness of the first pupil roundness and a true circular reference.

4. The system of claim 1 , wherein the control processor is to compute the first 3D origin point to include a first y-value and a first z-value as the first vertex of curvature.

5. The system of claim 1 , wherein the control processor is to compute the first 3D origin point to include a first y-value and a first z-value corresponding to an eyeball center of the first eye based on the first vertex of curvature and a radius of curvature of the first corneal surface estimated from the first imagery.

6. The system of claim 1 , further comprising:
a nose portion integrated with the wearable structure and configured to reflect illumination projected from the first illuminator back to the first imaging array.

7. The system of claim 1 , further comprising:
a nose portion integrated with the wearable structure and having a measurable feature thereon, the feature being recognizable to the control processor as having a pre-calibrated size,
wherein the control processor is to compute the first 3D origin point to include a first x-value by measuring a detected size of the feature in the first imagery and computing an x-distance from the first imaging array to the feature based on a difference between the detected size and the pre-calibrated size, and determining the first x-value based on the x-distance.

8. The system of claim 1 , further comprising:
a second illuminator integrated with the wearable structure at a second temporal side of a second eye of the user to project illumination across a second cornea of the second eye, the first and second temporal sides being on opposite sides of the user's head; and
a second imaging array integrated with the wearable structure at the second temporal side to generate second imagery of the second eye,
wherein the control processor is further to:
compute a second 3D origin point of the second eye based at least on detecting a second vertex of curvature of a second corneal surface of the second eye in the second imagery; and
compute a second angular coordinate measurement of the second eye based on detecting a second pupil center location and/or a second pupil roundness in the second imagery, and

wherein the control processor is to compute the sightline based further on the second 3D origin point and the second angular coordinate measurement.

9. The system of claim 8 , wherein the control processor is to compute the first 3D origin point to include a first x-value computed based on a center-point between the first imaging array and the second imaging array.

10. The system of claim 8 , further comprising:
a nose portion integrated with the wearable structure and positioned so that the first imaging array and the first illuminator are to a first side of the nose portion, and the second imaging array and the second illuminator are to a second side of the nose portion, the nose portion configured to mitigate optical crosstalk between the first side and the second side.

11. The system of claim 1 , wherein the control processor is integrated with the wearable structure.

12. The system of claim 1 , further comprising:
a display integrated with the wearable structure,
wherein the control processor is further to determine a display coordinate system based on the 3D origin point and to display one or more virtual objects on the display based on the display coordinate system and the sightline.

13. The system of claim 12 , further comprising:
a scene sensor integrated with the wearable structure to detect one or more features of a real-world environment in a real-world coordinate system,
wherein the control processor is further to compute a mapping between the display coordinate system to the real-world coordinate system and to display at least one of the virtual objects on the display as projected into the real-world environment based on the mapping and the sightline.

14. The system of claim 1 , wherein the first illuminator projects the illumination in a non-visible, eye-safe frequency band, and the first imaging array has receiving optics including a bandpass filter transmissive to the non-visible, eye-safe frequency band.

15. The system of claim 1 , wherein the wearable structure is integrated with eyeglass frames.

16. A method for optical sightline tracking, the method comprising:
projecting first illumination, by a first illuminator of a wearable structure located at a first temporal side of a first eye of a user wearing the wearable structure, the first illumination projected across a first cornea of the first eye;
generating first imagery of the first eye, under the first illumination, by a first imaging array of the wearable structure located at the first temporal side;
computing, by a control processor of the wearable structure, a first three-dimensional (3D) origin point of the first eye based at least on detecting a first vertex of curvature of a first corneal surface of the first eye in the first imagery;
computing, by the control processor, a first angular coordinate measurement of the first eye based on detecting a first set of pupil dimensions in the first imagery and computing an offset between the first set of pupil dimensions and a pupil dimensional reference; and
computing, by the control processor, a sightline based on the first 3D origin point and the first angular coordinate measurement.

17. The method of claim 16 , wherein the computing the first angular coordinate measurement of the first eye comprises detecting a first pupil center location in the first imagery and computing the offset as between the first pupil center location and the first vertex of curvature.

18. The method of claim 16 , wherein the computing the first angular coordinate measurement of the first eye comprises detecting a roundness of the first pupil and computing the offset as between the roundness of the first pupil roundness and a true circular reference.

19. The method of claim 16 , further comprising:
projecting secon

Assignees

• Shenzhen Goodix Tech Co Ltd

Inventors

• He Yi
• Pi Bo

Classifications

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G02B27/0093 Primary

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B27/017

  Head mounted · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title