Gaze direction mapping

Patent metadata

Field	Value
Publication number	US-9870051-B2
Application number	US-201715692714-A
Country	US
Kind code	B2
Filing date	Aug 31, 2017
Priority date	Aug 7, 2015
Publication date	Jan 16, 2018
Grant date	Jan 16, 2018

Abstract

A method for determining correspondence between a gaze direction and an environment around a wearable device is disclosed. The wearable device may include an eye tracking device and an outward facing image sensor. The method may include receiving an input parameter and at least one scene image from the outward facing image sensor. The method may further include determining, with at least the eye tracking device, at least one gaze direction of a wearer of the wearable device at a point in time corresponding to when the scene image was captured by the outward facing image sensor. The method may additionally include determining, based at least in part on the input parameter, that a particular scene image includes at least a portion of a predefined image. The method may moreover include determining, based on the at least one gaze direction, at least one gaze point on the particular scene image.

First claim

What is claimed is:

1. A method for determining a correspondence between a gaze direction and an environment around a wearable device, wherein the wearable device comprises an eye tracking device and an outward facing image sensor, and wherein the method comprises:
receiving an input parameter comprising a predefined image and dynamic area data identifying at least one dynamic area within the predefined image;
receiving a scene image from the outward facing image sensor;
determining, with at least the eye tracking device, a gaze direction of a wearer of the wearable device at a point in time corresponding to when the scene image was captured by the outward facing image sensor;
determining, based at least in part on the input parameter, that the scene image includes at least a remaining portion of the predefined image, wherein the remaining portion of the predefined image does not include the at least one dynamic area;
determining a position of the remaining portion in the scene image; and
determining, based on the at least the gaze direction and the position, a gaze point on the predefined image.

2. The method for determining a correspondence between a gaze direction and an environment around a wearable device of claim 1 , wherein the method further comprises:
receiving dynamic area content information comprising an image; and
determining, based at least on the dynamic area content information, that the particular scene image also includes at least a portion of the image in a location of the dynamic area in the scene image.

3. The method for determining a correspondence between a gaze direction and an environment around a wearable device of claim 2 , wherein:
the dynamic content information is correlated to the point in time corresponding to when the scene image was captured by the outward facing image sensor.

4. The method for determining a correspondence between a gaze direction and an environment around a wearable device of claim 1 , wherein the predefined image comprises:
predefined text.

5. The method for determining a correspondence between a gaze direction and an environment around a wearable device of claim 1 , wherein the input parameter comprises:
a manually mapped point in the predefined image.

6. The method for determining a correspondence between a gaze direction and an environment around a wearable device of claim 1 , wherein the input parameter comprises:
a manually mapped area of the predefined image.

7. The method for determining a correspondence between a gaze direction and an environment around a wearable device of claim 1 , wherein the input parameter comprises:
a manually identified presence of the predefined image in the scene image.

8. The method for determining a correspondence between a gaze direction and an environment around a wearable device of claim 1 , wherein the input parameter comprises:
a plurality of identified instances of the predefined image in video received from the outward facing image sensor.

9. A system for determining a correspondence between a gaze direction and an environment around a wearable device, wherein the system comprises:
one or more processors configured to at least:
receive an input parameter comprising a predefined image and dynamic area data identifying at least one dynamic area within the predefined image;
receive a scene image from an outward facing image sensor of a wearable device;
determine, with at least an eye tracking device of the wearable device, a gaze direction of a wearer of the wearable device at a point in time corresponding to when the scene image was captured by the outward facing image sensor;
determine, based at least in part on the input parameter, that the scene image includes at least a remaining portion of the predefined image, wherein the remaining portion of the predefined image does not include the at least one dynamic area;
determine a position of the remaining portion in the scene image; and
determine, based at least on the gaze direction and the position, a gaze point on the predefined image.

10. The system for determining a correspondence between a gaze direction and an environment around a wearable device of claim 9 , wherein the one or more processors are further configured to at least:
receive dynamic area content information comprising an image; and
determine, based at least on the dynamic area content information, that the particular scene image also includes at least a portion of the image in a location of the dynamic area in the scene image.

11. The system for determining a correspondence between a gaze direction and an environment around a wearable device of claim 10 , wherein:
the dynamic content information is correlated to the point in time corresponding to when the scene image was captured by the outward facing image sensor.

12. The system for determining a correspondence between a gaze direction and an environment around a wearable device of claim 9 , wherein the predefined image comprises:
predefined text.

13. The system for determining a correspondence between a gaze direction and an environment around a wearable device of claim 9 , wherein the input parameter comprises:
a manually mapped area in the predefined image.

14. The system for determining a correspondence between a gaze direction and an environment around a wearable device of claim 9 , wherein the input parameter comprises:
a manually mapped area of the predefined image.

15. A non-transitory machine readable medium having instructions stored thereon for determining a correspondence between a gaze direction and an environment around a wearable device, wherein the instructions are executable by one or more processors for at least:
receiving an input parameter comprising a predefined image and dynamic area data identifying at least one dynamic area within the predefined image;
receiving a scene image from an outward facing image sensor of a wearable device;
determining, with at least an eye tracking device of the wearable device, a gaze direction of a wearer of the wearable device at a point in time corresponding to when the scene image was captured by the outward facing image sensor;
determining, based at least in part on the input parameter, that the scene image includes at least a remaining portion of the predefined image, wherein the remaining portion of the predefined image does not include the at least one dynamic area;
determining a position of the remaining portion in the scene image; and
determining, based on the at least the gaze direction and the position, a gaze point on the predefined image.

16. The non-transitory machine readable medium of claim 15 , wherein the instructions are further executable for at least:
receiving dynamic area content information comprising an image; and
determining, based at least on the dynamic area content information, that the particular scene image also includes at least a portion of the image in a location of the dynamic area in the scene image.

17. The non-transitory machine readable medium of claim 16 , wherein:
the dynamic content information is correlated to the point in time corresponding to when the scene image was captured by the outward facing image sensor.

18. The non-transitory machine readable medium of claim 15 , wherein the predefined image comprises:
predefined text.

19. The non-transitory machine readable medium of claim 15 , wherein the input parameter comprises:
a manually mapped area in the predefined image.

20. The non-transitory m

Assignees

• Tobii Ab

Inventors

• Lovtjärn André
• Högström Jesper
• Högström Jonas
• Petersson Rasmus
• Skogö Mårten
• Wong Wilkey

Classifications

• G06F16/434

  using image data, e.g. images, photos, pictures taken by a user · CPC title

• G06F3/005

  Input arrangements through a video camera · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G06F3/04815

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title