Gestures, interactions, and common ground in a surface computing environment

Patent metadata

Field	Value
Publication number	US-10409381-B2
Application number	US-201514822456-A
Country	US
Kind code	B2
Filing date	Aug 10, 2015
Priority date	Dec 15, 2008
Publication date	Sep 10, 2019
Grant date	Sep 10, 2019

Abstract

Aspects relate to detecting gestures that relate to a desired action, wherein the detected gestures are common across users and/or devices within a surface computing environment. Inferred intentions and goals based on context, history, affordances, and objects are employed to interpret gestures. Where there is uncertainty in intention of the gestures for a single device or across multiple devices, independent or coordinated communication of uncertainty or engagement of users through signaling and/or information gathering can occur.

First claim

What is claimed is:

1. A method performed by one or more processors that are configured with executable instructions, the method comprising:
receiving, by a first computing device, a first gesture input from a touch-sensitive input device;
determining whether the first gesture input is an undefined gesture that is absent from a first set of gestures stored by the first computing device;
communicating, over a network, with a second computing device of a user to obtain a second set of gestures of the user stored by the second computing device, the second set of gestures including a plurality of gestures defined by the user, each gesture of the second set of gestures associated with a particular command;
automatically converting the second set of gestures into gestures to be used on the first computing device;
determining whether the first gesture input is a gesture included in the automatically converted second set of gestures;
determining a command associated with the first gesture input based on a gesture of the automatically converted second set of gestures when the first gesture input is determined to be a gesture included in the automatically converted second set of gestures; and
performing the determined command associated with the first gesture input.

2. The method of claim 1 , wherein:
the second computing device comprises a server; and
the network comprises an Internet.

3. The method of claim 1 , further comprising:
receiving a second gesture input within a predetermined time period after receiving the first gesture input;
determining whether the second gesture input is a duplicate gesture based on the first gesture input, wherein the first gesture input and the second gesture input are associated with a same software application; and
ignoring the duplicate gesture when the second gesture input is determined to be the duplicate gesture.

4. The method of claim 1 , further comprising:
determining whether a second gesture input is included in the first set of gestures;
determining a second command corresponding to the second gesture input based on the first set of gestures or the automatically converted second set of gestures; and
performing the determined second command corresponding to the second gesture input.

5. The method of claim 1 , further comprising:
determining whether a second gesture is absent from the first set of gestures; and
displaying a prompt to select a second command to be associated with the second gesture when the second gesture is absent from the first set of gestures;
receiving selection of the second command; and
associating the selected second command with the second gesture.

6. A first computing device comprising:
an input device;
one or more processors;
one or more memory devices to store instructions that are executable by the one or more processors to perform operations comprising:
receiving a first gesture input from the input device;
determining whether the first gesture input is absent from a first set of gestures stored in the one or more memory devices;
communicating, over a network, with a second computing device of a user to obtain a second set of gestures of the user stored by the second computing device, the second set of gestures including a plurality of gestures defined by the user, each gesture of the second set of gestures associated with a particular command;
automatically converting the second set of gestures into gestures to be used on the first computing device;
identifying a first command associated with the first gesture input based on a gesture of the automatically converted second set of gestures; and
sending the identified first command to a software application.

7. The first computing device of claim 6 , wherein the second computing device comprises a server.

8. The first computing device of claim 6 , wherein the network comprises an Internet.

9. The first computing device of claim 6 , wherein the operations further comprise:
receiving a second gesture input;
determining whether the second gesture input is included in the first set of gestures;
determining a second command is associated with the second gesture input based on a gesture of the first set of gestures when the second gesture input is included in the first set of gestures; and
performing the determine second command associated with the second gesture input.

10. The first computing device of claim 6 , wherein communicating, over the network, with the second computing device of the user to obtain the second set of gestures of the user stored by the second computing device comprises:
sending a request, over the network, from the first computing device to the second computing device to provide the second set of gestures; and
receiving, over the network, the second set of gestures from the second computing device.

11. The first computing device of claim 6 , wherein the operations further comprise:
receiving a second gesture input;
determining whether the second gesture input is absent from the first set of gestures;
receiving user input selecting a second command to be associated with the second gesture input when the second gesture is absent from the first set of gestures; and
associating the selected second command with the second gesture input.

12. The first computing device of claim 6 , wherein the operations further comprise:
tracking a gaze of the user that is using the first computing device; and
identifying, based on the gaze of the user, an item displayed on a display device, wherein sending the identified first command to the software application causes the software application to perform the identified first command on the item displayed on the display device.

13. A method comprising:
receiving, by a first computing device, a first gesture input from an input device associated with the first computing device;
determining whether the first gesture input is absent from a first set of gestures stored on the first computing device;
communicating, over a network, with a second computing device of a user to obtain a second set of gestures of the user stored by the second computing device, the second set of gestures including a plurality of gestures defined by the user, each gesture of the second set of gestures associated with a particular command;
automatically converting the second set of gestures into gestures to be used on the first computing device;
determining whether the first gesture input is a gesture included in the automatically converted second set of gestures;
determining a first command associated with the first gesture input based on a gesture of the automatically converted second set of gestures when the first gesture input is determined to be a gesture included in the automatically converted second set of gestures;
performing the determined first command associated with the first gesture input;
receiving a second gesture input from the input device within a predetermined time period from receiving the first gesture input;
determining whether the second gesture input is a duplicate gesture based on the first gesture input, wherein the first gesture input and the second gesture input are associated with a same software application executing on the first computing device; and
ignoring the second gesture input when the second gesture input is determined to be the duplicate gesture.

14. The method of claim 13 , wherein:
the second computing device comprises a server; and
the network comprises an Internet.

15. The method of claim 13 , wherein communicating, over the network, with the second computing devic

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Morris Meredith June
• Horvitz Eric J
• Wilson Andrew David
• Jones F David
• Hodges Stephen E
• Hinckley Kenneth P
• Butler David Alexander
• Sands Ian M
• Russ V Kevin
• Benko Hrvoje
• Leprowse Shawn R
• Izadi Shahram
• Kunz William Ben

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• H04L67/42

  Electricity · mapped topic