Gaze direction mapping

Patent metadata

Field	Value
Publication number	US-9829976-B2
Application number	US-201514954026-A
Country	US
Kind code	B2
Filing date	Nov 30, 2015
Priority date	Aug 7, 2015
Publication date	Nov 28, 2017
Grant date	Nov 28, 2017

Abstract

A method for determining correspondence between a gaze direction and an environment around a wearable device is disclosed. The wearable device may include an eye tracking device and an outward facing image sensor. The method may include receiving an input parameter and at least one scene image from the outward facing image sensor. The method may further include determining, with at least the eye tracking device, at least one gaze direction of a wearer of the wearable device at a point in time corresponding to when the scene image was captured by the outward facing image sensor. The method may additionally include determining, based at least in part on the input parameter, that a particular scene image includes at least a portion of a predefined image. The method may moreover include determining, based on the at least one gaze direction, at least one gaze point on the particular scene image.

First claim

What is claimed is:

1. A method for determining a correspondence between a gaze direction and an environment around a wearable device, wherein the wearable device comprises an eye tracking device and an outward facing image sensor, and wherein the method comprises:
receiving at least one scene image from the outward facing image sensor;
determining, with at least the eye tracking device, at least one gaze direction of a wearer of the wearable device at a point in time corresponding to when the scene image was captured by the outward facing image sensor;
determining, based at least on an input parameter, that a particular scene image includes at least a portion of a predefined image;
determining, based on the at least one gaze direction, at least one gaze point on the particular scene image;
causing the scene image to be displayed with an overlaid visual indication, wherein the overlaid visual indication corresponds to at least a portion of the predefined image;
receiving user feedback regarding adjustment of the overlaid visual indication;
adjusting the input parameter based at least in part on the user feedback; and
determining, based on the input parameter as adjusted, that a future scene image includes at least a portion of the predefined image.

2. The method according to claim 1 , wherein the input parameter comprises:
a manually mapped gaze point on the predefined image.

3. The method according to claim 1 , wherein the input parameter comprises
a manually mapped area of the predefined image.

4. The method according to claim 1 , wherein the input parameter comprises:
a manually mapped area within the predefined image.

5. The method according to claim 1 , wherein the input parameter comprises:
a manually provided indication that the predefined image exists in the scene image.

6. The method according to claim 1 , wherein the input parameter comprises:
a string of text found in the predefined image.

7. The method according to claim 1 , wherein the input parameter comprises:
a visual representation of the predefined image.

8. The method according to claim 1 , wherein the overlaid visual indication comprises:
a representation of an area of interest as determined based at least on the input parameter.

9. The method according to claim 1 , wherein the overlaid visual indication comprises:
a representation of an area in which the predefined image appears in the scene image.

10. The method according to claim 1 , wherein the method further comprises:
determining a confidence value which is a representation of how probable it is that the at least one gaze point relates to the predefined image.

11. The method according to claim 10 , wherein the method further comprises:
determining a quality value which is a representation of a degree of error in the determined at least one gaze point on the particular scene image.

12. The method according to claim 11 , wherein:
when the confidence value is high, and the quality value is low, determining, at least one gaze point comprises determining a gaze area.

13. The method according to claim 10 , wherein determining the confidence value comprises:
determining a change has occurred in an area of the particular scene image where the predefined image was present at at least one point in time.

14. The method according to claim 11 , wherein determining the quality value comprises:
determining a difference in capture rate between the image sensor and the eye tracking device.

15. A system for determining a correspondence between a gaze direction and an environment around a wearable device, wherein the system comprises:
a wearable device comprising an eye tracking device and an outward facing image sensor; and
a processor configured to at least:
receive at least one scene image from the outward facing image sensor;
determine, with at least the eye tracking device, at least one gaze direction of a wearer of the wearable device at a point in time corresponding to when the scene image was captured by the outward facing image sensor;
determine, based at least on an input parameter, that a particular scene image includes at least a portion of a predefined image;
determine, based on the at least one gaze direction, at least one gaze point on the particular scene image;
cause the scene image to be displayed with an overlaid visual indication, wherein the overlaid visual indication corresponds to at least a portion of the predefined image;
receive user feedback regarding adjustment of the overlaid visual indication;
adjust the input parameter based at least in part on the user feedback; and
determine, based on the input parameter as adjusted, that a future scene image includes at least a portion of the predefined image.

16. The system of claim 15 , wherein the processor is further configured to at least:
determine a confidence value which is a representation of how probable it is that the at least one gaze point relates to the predefined image.

17. The system of claim 16 , wherein the processor is further configured to at least:
determine a quality value which is a representation of a degree of error in the determined at least one gaze point on the particular scene image.

18. A non-transitory machine readable medium having instructions stored thereon for determining a correspondence between a gaze direction and an environment around a wearable device, wherein the instructions are executable by one or more processors to at least:
receive at least one scene image from the outward facing image sensor;
determine, with at least the eye tracking device, at least one gaze direction of a wearer of the wearable device at a point in time corresponding to when the scene image was captured by the outward facing image sensor;
determine, based at least on an input parameter, that a particular scene image includes at least a portion of a predefined image;
determine, based on the at least one gaze direction, at least one gaze point on the particular scene image;
cause the scene image to be displayed with an overlaid visual indication, wherein the overlaid visual indication corresponds to at least a portion of the predefined image;
receive user feedback regarding adjustment of the overlaid visual indication;
adjust the input parameter based at least in part on the user feedback; and
determine, based on the input parameter as adjusted, that a future scene image includes at least a portion of the predefined image.

19. The non-transitory machine readable medium of claim 18 , wherein the instructions are further executable by the one or more processors to at least:
determine a confidence value which is a representation of how probable it is that the at least one gaze point relates to the predefined image.

20. The non-transitory machine readable medium of claim 19 , wherein the instructions are further executable by the one or more processors to at least:
determine a quality value which is a representation of a degree of error in the determined at least one gaze point on the particular scene image.

Assignees

• Tobii Ab

Inventors

• Algotsson André
• Clausen Anders
• Högström Jesper
• Högström Jonas
• Lindgren Tobias
• Petersson Rasmus
• Skogö Mårten
• Wong Wilkey

Classifications

• G02B27/017

  Head mounted · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G06F17/30047

  Physics · mapped topic

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/005

  Input arrangements through a video camera · CPC title