Systems and methods for calibrating image sensors in wearable apparatuses

Patent metadata

Field	Value
Publication number	US-11665333-B2
Application number	US-202016917657-A
Country	US
Kind code	B2
Filing date	Jun 30, 2020
Priority date	Feb 15, 2018
Publication date	May 30, 2023
Grant date	May 30, 2023

Abstract

In one embodiment, a wearable apparatus is provided which may include a holographic film disposed on the wearable apparatus, an eye tracking device including an image sensor and an illuminator, and one or more processors. The processors may be configured to activate the illuminator to illuminate the holographic film and capture, with the image sensor, an image of at least a portion of the holographic film while the holographic film is illuminated. The processors may also be configured to determine a characteristic of the holographic film based on the image and determine, based on the characteristic, a location or an orientation of the image sensor relative to the holographic film. The processors may further be configured to change, based on the location or the orientation of the image sensor relative to the holographic film, at least one calibration parameter of the image sensor.

First claim

The invention claimed is:

1. An augmented reality, virtual reality, or other wearable apparatus comprising:
a holographic film disposed on a wearable apparatus;
an eye tracking device comprising an image sensor and an illuminator; and one or more processors configured to at least:
activate the illuminator to illuminate the holographic film, wherein the illuminator is arranged to create a glint at the eye;
capture, with the image sensor, an image of at least a portion of the holographic film while the holographic film is illuminated;
determine a characteristic of the holographic film based on the image;
determine, based on the characteristic of the holographic film, at least one of a location or an orientation of the image sensor relative to the holographic film; and
change, based on at least one of the location or the orientation of the image sensor relative to the holographic film, at least one calibration parameter of the image sensor.

2. The augmented reality, virtual reality, or other wearable apparatus of claim 1 , wherein the image comprises:
a reflection of light off the holographic film.

3. The augmented reality, virtual reality, or other wearable apparatus of claim 2 , wherein the reflection comprises:
a diffuse reflection from a feature of the holographic film.

4. The augmented reality, virtual reality, or other wearable apparatus of claim 2 , wherein the reflection comprises:
a specular reflection of a feature of the holographic film.

5. The augmented reality, virtual reality, or other wearable apparatus of claim 1 , wherein the image comprises:
a diffuse reflection of light surrounding an area having no reflection of light.

6. The augmented reality, virtual reality, or other wearable apparatus of claim 1 , wherein the image comprises:
a diffuse reflection of light bounded by an edge of the holographic film.

7. The augmented reality, virtual reality, or other wearable apparatus of claim 1 , wherein determining the location or the orientation of the image sensor relative to the holographic film comprises:
determining the location or the orientation of the image sensor in relation to three orthogonal axes.

8. The augmented reality, virtual reality, or other wearable apparatus of claim 7 , wherein the holographic film comprises:
three characteristics disposed in two axes in the holographic film which specularly reflect light toward the image sensor.

9. The augmented reality, virtual reality, or other wearable apparatus of claim 7 , wherein the holographic film comprises:
only one characteristic which reflects light, wherein the characteristic is two-dimensional.

10. The augmented reality, virtual reality, or other wearable apparatus of claim 7 , wherein the holographic film comprises:
at least one characteristic which does not reflect light, wherein at least some portion of the at least one characteristic is bounded by reflective portions of the holographic film, and wherein the at least one characteristic is two-dimensional.

11. A method for changing at least one calibration parameter of an image sensor in an eye tracking device of an augmented reality, virtual reality, or other wearable apparatus, the method comprising:
activating an illuminator of an eye tracking device to illuminate a holographic film disposed on a wearable apparatus, wherein the illuminator is arranged to create a glint at the eye;
capturing, with an image sensor of the eye tracking device, an image of at least a portion of the holographic film while the holographic film is illuminated;
determining a characteristic of the holographic film based on the image;
determining, based on the characteristic of the holographic film, at least one of a location or an orientation of the image sensor relative to the holographic film; and
changing, based on the location or the orientation of the image sensor relative to the holographic film, at least one calibration parameter of the image sensor.

12. The method for changing at least one calibration parameter of an image sensor in an eye tracking device of an augmented reality, virtual reality, or other wearable apparatus of claim 11 , wherein the image comprises:
a reflection of light off the holographic film, and wherein the reflection comprises a diffuse reflection from a feature of the holographic film or a specular reflection of a feature of the holographic film.

13. The method for changing at least one calibration parameter of an image sensor in an eye tracking device of an augmented reality, virtual reality, or other wearable apparatus claim 11 , wherein the image comprises:
a diffuse reflection of light surrounding an area having no reflection of light.

14. The method for changing at least one calibration parameter of an image sensor in an eye tracking device of an augmented reality, virtual reality, or other wearable apparatus of claim 11 , wherein the image comprises:
a diffuse reflection of light bounded by an edge of the holographic film.

15. The method for changing at least one calibration parameter of an image sensor in an eye tracking device of an augmented reality, virtual reality, or other wearable apparatus of claim 11 , wherein determining the location or the orientation of the image sensor relative to the holographic film comprises:
determining the location or the orientation of the image sensor in relation to three orthogonal axes.

16. A non-transitory machine-readable medium having instructions stored thereon for changing at least one calibration parameter of an image sensor in an eye tracking device of an augmented reality, virtual reality, or other wearable apparatus, wherein the instructions are executable by one or more processors to at least:
activate an illuminator of an eye tracking device to illuminate a holographic film disposed on a wearable apparatus, wherein the illuminator is arranged to create a glint at the eye;
capture, with an image sensor of the eye tracking device, an image of at least a portion of the holographic film while the holographic film is illuminated;
determine a characteristic of the holographic film based on the image;
determine, based on the characteristic of the holographic film, at least one of a location or an orientation of the image sensor relative to the holographic film; and
change, based on the location or the orientation of the image sensor relative to the holographic film, at least one calibration parameter of the image sensor.

17. The non-transitory machine-readable medium of claim 16 , wherein the image comprises:
a reflection of light off the holographic film, and wherein the reflection comprises a diffuse reflection from a feature of the holographic film or a specular reflection of a feature of the holographic film.

18. The non-transitory machine-readable medium of claim 16 , wherein the image comprises:
a diffuse reflection of light surrounding an area having no reflection of light.

19. The non-transitory machine-readable medium of claim 16 , wherein the image comprises:
a diffuse reflection of light bounded by an edge of the holographic film.

20. The non-transitory machine-readable medium of claim 16 , wherein determining the location or the orientation of the image sensor relative to the holographic film comprises:
determining the location or the orientation of the image sensor in relation to three orthogonal axes.

Assignees

• Tobii Ab

Inventors

• Tornéus Daniel
• Sundberg Torbjörn
• Arvidsson Magnus

Classifications

• G02B27/0093 Primary

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• H04N23/56

  provided with illuminating means · CPC title

• G06V40/19

  Sensors therefor · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title