System for gaze interaction

Patent metadata

Field	Value
Publication number	US-10488919-B2
Application number	US-201514985954-A
Country	US
Kind code	B2
Filing date	Dec 31, 2015
Priority date	Jan 4, 2012
Publication date	Nov 26, 2019
Grant date	Nov 26, 2019

Abstract

A method and system for assisting a user when interacting with a graphical user interface combines gaze based input with gesture based user commands. A user of a computer system without a traditional touch-screen can interact with graphical user interfaces in a touch-screen like manner using a combination of gaze based input and gesture based user commands. A solution for touch-screen like interaction uses gaze input and gesture based input as a complement or an alternative to touch-screen interactions with a computer device having a touch-screen. Combined gaze and gesture based interaction with graphical user interfaces can be used to achieve a touchscreen like environment in computer systems without a traditional touchscreen or in computer systems having a touchscreen arranged ergonomically unfavorable for the user or a touchscreen arranged such that it is more comfortable for the user to use gesture and gaze for the interaction than the touchscreen.

First claim

What is claimed is:

1. A method for providing interaction on a portable device, wherein the method comprises:
determining that the portable device is emitting audio content;
determining a gaze direction of an operator of the portable device, relative to the portable device;
determining that the gaze direction of the operator indicates that a gaze of the operator is outside of the portable device and has shifted towards a point that is outside of the portable device and that is within a predetermined distance from the portable device, the predetermined distance defined based on input of the operator of the portable device;
determining, based on the gaze being outside of the portable device and having shifted towards the portable device within the predetermined distance, a preferred method of audio change and parameters associated with the audio change, the preferred method comprising an audio notification different from the audio content, the audio notification signaling to the operator that the portable device recognizes a change in the gaze and that the portable device is dismissing the audio content, the parameters specifying an emission of the audio notification and a gradual change to the audio content; and
causing the audio content emitted by the portable device to be changed and the audio notification to be presented based on the preferred method of audio change and the parameters.

2. The method of claim 1 , wherein the audio content comprises at least one selection from a group consisting of:
music;
video sound;
a ring tone;
an alert; and
a warning.

3. The method of claim 1 , wherein determining the gaze direction of the operator comprises:
determining the gaze direction of the operator with an eye tracking device of the portable device.

4. The method of claim 1 , wherein determining the gaze direction of the operator comprises:
determining the gaze direction of the operator with an image sensor of the portable device.

5. The method of claim 1 , wherein the predetermined distance is five centimeters.

6. The method of claim 1 , wherein causing the audio content emitted by the portable device to be changed comprises a selection from a group consisting of:
gradually decreasing a volume of the audio content to a predetermined level;
gradually decreasing the volume of the audio content until the audio content is completely muted; and
changing the audio content from a first audio content to a second audio content.

7. The method of claim 6 , wherein causing the audio content emitted by the portable device to be changed comprises changing the audio content from a first audio content to a second audio content.

8. A non-transitory machine readable medium having instructions stored thereon for providing interaction on a portable device, wherein the instructions are executable by one or more processors for at least:
determining that the portable device is emitting audio content;
determining a gaze direction of an operator of the portable device, relative to the portable device;
determining that the gaze direction of the operator indicates that a gaze of the operator is outside of the portable device and has shifted towards a point that is outside of the portable device and that is within a predetermined distance from the portable device, the predetermined distance defined based on input of the operator of the portable device;
determining, based on the gaze direction of the operator, a preferred method of audio change and parameters associated with the audio change, the preferred method comprising an audio notification different from the audio content, the audio notification signaling to the operator that the portable device recognizes a change in the gaze and that the portable device is dismissing the audio content, the parameters specifying an emission of the audio notification and a change to the audio content; and
causing the audio content emitted by the portable device to be changed and the audio notification to be presented based on the preferred method of audio change and the parameters.

9. The non-transitory machine readable medium of claim 8 , wherein determining the gaze direction of the operator comprises:
determining the gaze direction of the operator with an eye tracking device of the portable device.

10. The non-transitory machine readable medium of claim 8 , wherein the predetermined distance is five centimeters.

11. The non-transitory machine readable medium of claim 8 , wherein causing the audio content emitted by the portable device to be changed comprises a selection from a group consisting of:
completely muting the audio content;
gradually decreasing a volume of the audio content to a predetermined level;
gradually decreasing the volume of the audio content until the audio content is completely muted; and
changing the audio content from a first audio content to a second audio content.

12. The non-transitory machine readable medium of claim 11 , wherein causing the audio content emitted by the portable device to be changed comprises changing the audio content from a first audio content to a second audio content.

13. The non-transitory machine readable medium of claim 8 , wherein determining the gaze direction of the operator comprises:
determining the gaze direction of the operator with an image sensor of the portable device.

14. A system for providing interaction on a portable device, wherein the system comprises:
the portable device configured to:
determine that the portable device is emitting audio content;
determine a gaze direction of an operator of the portable device, relative to the portable device;
determine that the gaze direction of the operator indicates that a gaze of the operator is outside of the portable device and has shifted towards a point that is outside of the portable device and that is within a predetermined distance from the portable device, the predetermined distance defined based on input of the operator of the portable device; and
determine, based on the gaze being outside of the portable device and having shifted towards the portable device within the predetermined distance, a preferred method of audio change and parameters associated with the audio change, the preferred method comprising an audio notification different from the audio content, the audio notification signaling to the operator that the portable device recognizes a change in the gaze and that the portable device is dismissing the audio content, the parameters specifying an emission of the audio notification and a change to the audio content; and
cause the audio content emitted by the portable device to be changed and the audio notification to be presented based on the preferred method of audio change and the parameters.

15. The system of claim 14 , wherein determining the gaze direction of the operator comprises:
determining the gaze direction of the operator with an eye tracking device or an image sensor of the portable device.

16. The system of claim 14 , wherein the predetermined distance is five centimeters.

17. The system of claim 14 , wherein causing the audio content emitted by the portable device to be changed comprises a selection from a group consisting of:
completely muting the audio content;
gradually decreasing a volume of the audio content to a predetermined level;
gradually decreasing the volume of the audio content until the audio content is completely muted; and
changing the audio content from a first audio content to a second audio content.

18. The system of claim 17 , wherein ca

Assignees

• Tobii Ab

Inventors

• George-Svahn Erland
• Dzemidzic Dzenan
• Vennström Anders
• Nilsson Ida
• Olsson Anders
• Lannsjö Rebecka
• Skogö Mårten

Classifications

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F1/1686

  the I/O peripheral being an integrated camera · CPC title

• G02B27/017

  Head mounted · CPC title