Eye tracker based contextual action

Patent metadata

Field	Value
Publication number	US-9507418-B2
Application number	US-69128910-A
Country	US
Kind code	B2
Filing date	Jan 21, 2010
Priority date	Jan 21, 2010
Publication date	Nov 29, 2016
Grant date	Nov 29, 2016

Abstract

The present invention relates to systems and methods for assisting a user when interacting with a graphical user interface by combining eye based input with input for e.g. selection and activation of objects and object parts and execution of contextual actions related to the objects and object parts. The present invention also relates to such systems and methods in which the user can configure and customize specific combinations of eye data input and input that should result in a specific contextual action.

First claim

That which is claimed is:

1. A method for manipulating objects or part of one of the objects and performing contextual actions related to the objects presented on a display of a computer device associated with an eye tracking unit, the computer device configured to store a plurality of predefined contextual actions to be executed for each combination of an eye tracking data signal and one of a plurality of predefined activation inputs, the method comprising:
displaying objects on the display of the computer device;
generating, from the eye tracking unit, an eye-tracking data signal identifying a user's gaze point to one of the displayed objects or part of one of the displayed objects on the display;
receiving, from a mechanical input device and based on a user's selection, one of the plurality of predefined activation inputs;
determining a specific contextual action based on the eye tracking data signal identifying the one of the displayed objects or the part of one of the displayed objects in combination with the user's selected predefined activation input, with the predefined activation input being configured as an input signal indicating a continuously maintained activation of the mechanical input device, and determining the specific contextual action is performed in response to releasing the continuously maintained input of the mechanical input device, and wherein different predefined activation inputs for a same object or object part of interest result in determining different specific contextual actions; and
executing the specific contextual action.

2. The method of claim 1 , wherein the specific contextual action is determined by a list identifying contextual actions to be executed for each combination of object or object part and activation input.

3. The method of claim 1 , further comprising:
configuring which specific contextual action should result from a specific combination of object or object part of interest on which the user is gazing and received activation input.

4. The method of claim 1 , wherein determining a specific contextual action is further based on a historical or current state of the object or object part of interest.

5. The method of claim 1 , further comprising:
showing the user's gaze point on the display to indicate for the user which object or object part that will be subjected to the contextual action.

6. The method of claim 1 , wherein said contextual action is determined by a probability calculation.

7. The method of claim 6 , wherein said probability calculation selects the object or object part geometrically closest to the user's gaze point.

8. The method of claim 6 , wherein said probability calculation is based on stored user patterns.

9. The method of claim 6 , wherein said probability calculation is based on a state of an object or of an object part of interest.

10. The method of claim 6 , wherein said probability calculation is based on a historical or current state of the computer device.

11. The method of claim 1 , wherein the mechanical input device is a keyboard.

12. The method of claim 1 , wherein the mechanical input device is a mouse button.

13. The method of claim 1 , wherein the mechanical input device is a touch pad.

14. A system for assisting a user in manipulating objects or part of one of the objects and performing contextual actions related to the objects presented on a display of a computer device associated with an eye tracking unit capable of determining user's gaze, the system comprising:
a display;
a computer device configured to generate objects on said display;
an eye tracking unit configured to generate an eye tracking data signal identifying a user's gaze point to one of the generated objects or part of one of the generated objects on said display; and
a mechanical input device configured to generate, based on a user's selection, one of a plurality of predefined activation inputs;
said computer device configured to:
store a plurality of predefined contextual actions to be executed for each combination of the eye tracking data signal and one of the plurality of predefined activation inputs,
determine a specific contextual action based on the eye tracking data signal identifying the one of the displayed objects or the part of one of the displayed objects the part of the displayed objects in combination with the user's selected predefined activation input, with the predefined activation input being configured as an input signal indicating a continuously maintained activation of the mechanical input device, and determining the specific contextual action is performed in response to releasing the continuously maintained input of the mechanical input device, and wherein different predefined activation inputs for a same object or object part of interest result in determining different specific contextual actions, and
execute the specific contextual action.

Assignees

• Yu Aron
• Elvesjo John Mikael Holtz
• Tobii Ab

Inventors

• Yu Aron
• Elvesjo John Mikael Holtz

Classifications

• G06F3/038

  Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry · CPC title

• G06F3/03543

  Mice or pucks (G06F3/03541 takes precedence) · CPC title

• G06F3/0488

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/04845

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title