System for gaze interaction

Patent metadata

Field	Value
Publication number	US-10013053-B2
Application number	US-201213646299-A
Country	US
Kind code	B2
Filing date	Oct 5, 2012
Priority date	Jan 4, 2012
Publication date	Jul 3, 2018
Grant date	Jul 3, 2018

Abstract

A method and system for assisting a user when interacting with a graphical user interface combines gaze based input with gesture based user commands. A user of a computer system without a traditional touch-screen can interact with graphical user interfaces in a touch-screen like manner using a combination of gaze based input and gesture based user commands. A solution for touch-screen like interaction uses gaze input and gesture based input as a complement or an alternative to touch-screen interactions with a computer device having a touch-screen. Combined gaze and gesture based interaction with graphical user interfaces can be used to achieve a touchscreen like environment in computer systems without a traditional touchscreen or in computer systems having a touchscreen arranged ergonomically unfavorable for the user or a touchscreen arranged such that it is more comfortable for the user to use gesture and gaze for the interaction than the touchscreen.

First claim

The invention claimed is:

1. A control module for executing object selection commands during user interaction with an information presentation area displaying an object, wherein the control module is configured to:
acquire user gaze data signals from a gaze tracking module, the user gaze signals indicating a gaze point on the information presentation area;
receive, from an input device that includes a physical touch sensitive area, first user gesture data corresponding to a press of a finger on the physical touch sensitive area;
generate a gaze point area on the information presentation area based on the user gaze data signals and on the first user gesture data, the gaze point area being a local area that includes the gaze point;
receive, from the input device, second user gesture data corresponding to a movement of the finger on the physical touch sensitive area, the press of the finger followed next by the movement of the finger;
determine, based on the gaze point area, that the second user gesture data is associated with a fine tuning command to update the gaze point area;
resize, based on the fine tuning command, the gaze point area according to the second user gesture data to generate an updated gaze point area, the updated gaze point area including the object and overlapping with the gaze point area;
receive, from the input device, third user gesture data indicating a release of the finger from the physical touch sensitive area, the movement of the finger followed next by the release of the finger;
determine, based on the object being included in the updated gaze point area, that the third user gesture data is associated with a select command to select the object; and
select, based on the select command, the object in the updated gaze point area.

2. A method for executing object selection commands during user interaction with an information presentation area associated with a computer device, said method comprising:
acquiring gaze data signals from a gaze tracking module, the user gaze signals indicating a gaze point on the information presentation area;
receiving, from an input device that includes a physical touch sensitive area, first user gesture data corresponding to a press of a finger on the physical touch sensitive area;
generating a gaze point area on the information presentation area based on the user gaze data signals and on the first user gesture data, the gaze point area being a local area that includes the gaze point;
receiving, from the input device, second user gesture data corresponding to a movement of the finger on the physical touch sensitive area, the press of the finger followed next by the movement of the finger;
determining, based on the gaze point area, that the second user gesture data is associated with a fine tuning command to update the gaze point area;
resizing, based on the fine tuning command, the gaze point area according to the second user gesture data to generate an updated gaze point area, the updated gaze point area including an object and overlapping with the gaze point area;
receiving, from the input device, third user gesture data indicating a release of the finger from the physical touch sensitive area, the movement of the finger followed next by the release of the finger;
determining, based on the object being included in the updated gaze point area, that the third user gesture data is associated with a select command to select the object; and
selecting, based on the select command, the object in the updated gaze point area.

3. A system for user interaction with an information presentation area, said system comprising:
an input device comprising a physical touch sensitive area, said user input device being adapted to detect user gestures in a form of user finger touch on, movement on and release from the physical touch sensitive area;
a gaze tracking module adapted to detect gaze data of a user of said information presentation area;
a control module configured to:
acquire user gaze data signals from the gaze tracking module, the user gaze signals indicating a gaze point on the information presentation area;
receive, from the input device, first user gesture data corresponding to a press of a finger on the physical touch sensitive area;
generate a gaze point area on the information presentation area based on the user gaze data signals and on the first user gesture data, the gaze point area being a local area that includes the gaze point;
receive, from the input device, second user gesture data corresponding to a movement of the finger on the physical touch sensitive area, the press of the finger followed next by the movement of the finger;
determine, based on the gaze point area, that the second user gesture data is associated with a fine tuning command to update the gaze point area;
resize, based on the fine tuning command, the gaze point area according to the second user gesture data to generate an updated gaze point area, the updated gaze point area including the object and overlapping with the gaze point area;
receive, from the input device, third user gesture data indicating a release of the finger from the physical touch sensitive area, the movement of the finger followed next by the release of the finger;
determine, based on the object being included in the updated gaze point area, that the third user gesture data is associated with a select command to select the object; and
select, based on the select command, the object in the updated gaze point area.

4. A computer device associated with an information presentation area, said computer device comprising:
an input device comprising a physical touch sensitive area, said user input device being adapted to detect user gestures in a form of user finger touch on, movement on and release from the physical touch sensitive area;
a gaze tracking module adapted to detect gaze data of a user of said information presentation area;
a control module configured to:
acquire user gaze data signals from the gaze tracking module, the user gaze signals indicating a gaze point on the information presentation area;
receive, from the input device, first user gesture data corresponding to a press of a finger on the physical touch sensitive area;
generate a gaze point area on the information presentation area based on the user gaze data signals and on the first user gesture data, the gaze point area being a local area that includes the gaze point;
receive, from the input device, second user gesture data corresponding to a movement of the finger on the physical touch sensitive area, the press of the finger followed next by the movement of the finger;
determine, based on the gaze point area, that the second user gesture data is associated with a fine tuning command to update the gaze point area;
resize, based on the fine tuning command, the gaze point area according to the second user gesture data to generate an updated gaze point area, the updated gaze point area including the object and overlapping with the gaze point area;
receive, from the input device, third user gesture data indicating a release of the finger from the physical touch sensitive area, the movement of the finger followed next by the release of the finger;
determine, based on the object being included in the updated gaze point area, that the third user gesture data is associated with a select command to select the object; and
select, based on the select command, the object in the updated gaze point area.

5. A handheld portable device including an information presentation area and comprising an input device comprising a physical touch sensitive area, said user input device being adapted to detect user gestures in a form of user finger touch on, movement on and release from the physical touch sensitive area, and a gaze tracki

Assignees

• Tobii Ab

Inventors

• Cederlund Markus
• Gavelin Robert
• Vennström Anders
• Kaplan Anders
• Olsson Anders
• Skogö Mårten

Classifications

• G06F3/04883

  for inputting data by handwriting, e.g. gesture or text · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G06F2203/0381

  Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title