Eye-tracking enabled wearable devices

Patent metadata

Field	Value
Publication number	US-9958941-B2
Application number	US-201615276592-A
Country	US
Kind code	B2
Filing date	Sep 26, 2016
Priority date	Sep 24, 2015
Publication date	May 1, 2018
Grant date	May 1, 2018

Abstract

A method for mapping an input device to a virtual object in virtual space displayed on a display device is disclosed. The method may include determining, via an eye tracking device, a gaze direction of a user. The method may also include, based at least in part on the gaze direction being directed to a virtual object in virtual space displayed on a display device, modifying an action to be taken by one or more processors in response to receiving a first input from an input device. The method may further include, thereafter, in response to receiving the input from the input device, causing the action to occur, wherein the action correlates the first input to an interaction with the virtual object.

First claim

What is claimed is:

1. A method for mapping an input device to a virtual object in virtual space displayed on a display device, wherein the method comprises:
associating a first action to be taken by one or more processors in response to an input received from a user at the input device, the first action causing a first function;
determining, via an eye tracking device, that a gaze direction of the user is directed to a predefined portion of a virtual object having a virtual surface in virtual space displayed on a display device, the predefined portion being associated with modifying the first action to define a second action;
based at least in part on determining that the gaze direction of the user is directed to the predefined portion of the virtual object, modifying the first action to define the second action to be taken by one or more processors in response to receiving the input from the input device such that the second action, when performed, causes a second function comprising movement of a visual indicator across the virtual surface of the virtual object in a direction corresponding to a direction of the input at the input device; and
thereafter, in response to receiving the input from the input device, causing the second action associated with the input to occur.

2. The method for mapping the input device to a virtual object in virtual space displayed on a display device of claim 1 , wherein:
the eye tracking device and the display device are disposed within a wearable device.

3. The method for mapping the input device to a virtual object in virtual space displayed on a display device of claim 1 , wherein modifying the first action to be taken is further based at least in part on:
the gaze direction being directed to the virtual object for at least a predefined period of time.

4. The method for mapping the input device to a virtual object in virtual space displayed on a display device of claim 1 , wherein modifying the first action to be taken is further based at least in part on:
the gaze direction being directed to the virtual object while a second input is received.

5. The method for mapping the input device to a virtual object in virtual space displayed on a display device of claim 1 , wherein the method further comprises:
in response to modifying the first action to be taken, causing an indication to be provided to the user that the first action to be taken has been modified.

6. The method for mapping the input device to a virtual object in virtual space displayed on a display device of claim 1 , wherein the second action comprises:
a specialized action in the virtual space associated with the virtual object.

7. The method for mapping the input device to a virtual object in virtual space displayed on a display device of claim 1 , wherein:
the visual indicator comprises a pointer or other virtual location marker.

8. A non-transitory machine readable medium having instructions stored thereon for mapping an input device to a virtual object in virtual space displayed on a display device, wherein the instructions are executable by one or more processors to at least:
associate an action to be taken by one or more processors in response to an input received from a user at the input device;
determine, via an eye tracking device, that a gaze direction of the user is directed to a predefined portion of a virtual object in virtual space displayed on a display device, the predefined portion being associated with modifying the action;
based at least in part on determining that the gaze direction of the user is directed to the predefined portion of the virtual object, modify the action to be taken by the one or more processors in response to receiving the input from the input device such that the action, as modified, causes the virtual object to rotate in virtual space in a direction that corresponds to and is the same as a direction of the input at the input device; and
thereafter, in response to receiving the input from the input device, cause the action associated with the input, as modified, to occur.

9. The non-transitory machine readable medium of claim 8 , wherein modifying the action to be taken is further based at least in part on:
the gaze direction being directed to the virtual object for at least a predefined period of time.

10. The non-transitory machine readable medium of claim 8 , wherein modifying the action to be taken is further based at least in part on:
the gaze direction being directed to the virtual object while a second input is received.

11. The non-transitory machine readable medium of claim 8 , wherein the action as modified comprises:
a specialized action in the virtual space associated with the virtual object.

12. The non-transitory machine readable medium of claim 8 , wherein the input device comprises:
a mouse.

13. A system for mapping an input device to a virtual object in virtual space displayed on a display device, wherein the system comprises:
an eye tracking device for determining a gaze direction of a user;
one or more processors configured to at least:
associate a first action to be taken by one or more processors in response to an input received from the user at the input device, the first action comprising a first function;
determine, via the eye tracking device, that a gaze direction of the user is directed to a predefined portion of a virtual object comprising a virtual panel having virtual inputs displayed on a display device, the predefined portion being associated with modifying the first action to define a second action;
based at least in part on determining that the gaze direction of the user is directed to the predefined portion of the virtual object, modify the first action to define the second action to be taken by one or more processors in response to receiving the input from the input device such that the second action, when performed, causes a second function comprising actuation of the virtual inputs of the virtual control panel according to correlated inputs at the input device; and
thereafter, in response to receiving the input from the input device, cause the second action associated with the input to occur.

14. The system for mapping the input device to a virtual object in virtual space displayed on a display device of claim 13 , wherein modifying the first action to be taken is further based at least in part on:
the gaze direction being directed to the virtual object for at least a predefined period of time.

15. The system for mapping the input device to a virtual object in virtual space displayed on a display device of claim 13 , wherein modifying the first action to be taken is further based at least in part on:
the gaze direction being directed to the virtual object while a second input is received.

16. The system for mapping the input device to a virtual object in virtual space displayed on a display device of claim 13 , wherein the second action comprises:
a specialized action in the virtual space associated with the virtual object.

17. The system for mapping the input device to a virtual object in virtual space displayed on a display device of claim 13 , wherein:
the eye tracking device and the display device are disposed within a wearable device.

18. The system for mapping the input device to a virtual object in virtual space displayed on a display device of claim 13 , wherein the one or more processors are further configured to at least:
in response to modifying the first action to be taken, causing an indication to be provided to the user that t

Assignees

• Tobii Ab

Inventors

• Gustafsson Simon
• Bezugly Alexey
• Kingbäck Anders
• Clausen Anders

Classifications

• G09G2354/00

  Aspects of interface with display user · CPC title

• G09G2340/0464

  Positioning · CPC title

• G06V40/19

  Sensors therefor · CPC title

• G06F2203/0381

  Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer · CPC title

• G06F3/04815 Primary

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title