Gesture recognizer system architecture

Patent metadata

Field	Value
Publication number	US-9280203-B2
Application number	US-201113196618-A
Country	US
Kind code	B2
Filing date	Aug 2, 2011
Priority date	Jan 30, 2009
Publication date	Mar 8, 2016
Grant date	Mar 8, 2016

Abstract

Systems, methods and computer readable media are disclosed for a gesture recognizer system architecture. A recognizer engine is provided, which receives user motion data and provides that data to a plurality of filters. A filter corresponds to a gesture, that may then be tuned by an application receiving information from the gesture recognizer so that the specific parameters of the gesture—such as an arm acceleration for a throwing gesture—may be set on a per-application level, or multiple times within a single application. Each filter may output to an application using it a confidence level that the corresponding gesture occurred, as well as further details about the user motion data.

First claim

What is claimed:

1. A method for providing recognition of gestures made by a user using a gesture filter representing a gesture, comprising:
receiving data indicative of a user motion or pose, the data being captured by a camera;
determining an output of the gesture filter based on the data;
determining, based on the data, to send the output to a first application of a plurality of applications in response to determining that the data corresponds to the first application and does not correspond to a system application of the plurality of applications; and
sending the output to the first application of the plurality of applications.

2. The method of claim 1 , further comprising:
receiving second data indicative of a user motion or pose, the second data being captured by the camera;
determining a second output of the gesture filter based on the second data; and
sending the second application the second output when the second data corresponds to a second application of the plurality of applications, the first and second applications executing concurrently.

3. The method of claim 2 , wherein the gesture filter comprises a parameter, and further comprising:
receiving from the first application a first value for the parameter;
wherein determining the output of the gesture filter based on the data comprises:
determining the output based on the first value for the parameter;

receiving from the second application a second value for the parameter; and
wherein determining the second output of the gesture filter based on the second data comprises:
determining the second output based on the second value for the parameter.

4. The method of claim 1 , further comprising:
receiving second data indicative of a user motion or pose, the second data being captured by the camera;
determining a second output of the gesture filter based on the data; and
sending the system application the output when the second data corresponds to the system application of the plurality of applications, the data being indicative of a command issued to the system application, the system application and the first application executing concurrently.

5. The method of claim 4 , wherein sending the first application the output when the data is indicative of a command issued to a system-software application comprises:
sending the first application the output when the data is indicative of initializing execution of a user application, navigating up or down in a hierarchical list, opening a file, closing a file, saving a file, ending execution of a user software, pausing the user software, saving a state of the user software, or controlling an aspect of an operating system.

6. The method of claim 1 , wherein sending the first application the output when the data corresponds to a first application of a plurality of applications comprises:
sending the first application the output when the data is indicative of a command issued to user software.

7. The method of claim 6 , wherein sending the first application the output when the data is indicative of a command issued to user software comprises:
sending the first application the output when the data is indicative of manipulating an item or avatar in the first application.

8. The method of claim 1 , wherein, sending the first application the output when the data corresponds to a first application of a plurality of applications comprises:
sending the first application the output when the data is captured by the camera; and further comprising:
receiving second data indicative of sound captured by a microphone; and
sending an indication of the second data to a second application of the plurality of applications.

9. The method of claim 1 , wherein the gesture filter comprises a parameter, and further comprising:
receiving from the first application a value for the parameter; and
wherein, determining an output of the gesture filter based on the data comprises:
determining the output of the gesture filter based on the value of the parameter.

10. The method of claim 9 , wherein a parameter comprises an indication of a body part, a volume of space, a velocity, a direction of movement, an angle, or a place where a movement occurs.

11. A system for providing recognition of gestures made by a user using a gesture filter representing a gesture comprising:
a processor; and
a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, when executed by the processor, cause the system at least to:
receive data indicative of a user motion or pose, the data being captured by a camera;
determine an output of the gesture filter based on the data;
determine, based on the data, to send the output to a first application of a plurality of applications in response to determining that the data corresponds to the first application and does not correspond to a system application of the plurality of applications; and
send the output to the first application of the plurality of applications.

12. The system of claim 11 , wherein the gesture filter has a plurality of contexts, and a parameter of each context of the base information about the gesture is unique.

13. The system of claim 11 , wherein the memory further bears processor-executable instructions that, when executed by the processor, cause the system at least to:
receive the gesture filter from the first application.

14. The system of claim 11 , wherein the gesture filter comprises a parameter, and wherein the memory further bears processor-executable instructions that, when executed by the processor, cause the system at least to:
set a value for the parameter in response to receiving data captured by the depth camera indicative of a change in the user's fatigue, a change in the user's competence, or a second user replacing the user; and
wherein the processor-executable instructions that, when executed by the system, cause the system to at least determine an output of the gesture filter based on the data further cause the system to at least:
determine an output of the gesture filter based on the value for the parameter.

15. The system of claim 11 , wherein the gesture filter comprises a parameter, and wherein the memory further bears processor-executable instructions that, when executed by the processor, cause the system at least to:
in response to receiving second data indicative of a user motion or pose, the data being captured by the camera, receive from the first application a value for the parameter; and
wherein the processor-executable instructions that, when executed by the system, cause the system to at least determine an output of the gesture filter based on the data further cause the system to at least:
determine an output of the gesture filter based on the value for the parameter.

16. The system of claim 11 , wherein processor-executable instructions that, when executed by the processor, cause the system to at least determine the output of the gesture filter based on the data further causes the system at least to:
determine a confidence level that the data is indicative of the gesture being performed.

17. The system of claim 11 , wherein the gesture comprises:
a crouch, a jump, a lean, an arm throw, a toss, a swing, a dodge, a kick, or a block.

18. The system of claim 11 , wherein a gesture filter comprises a type of input and a type of output, and wherein the memory further bears processor-executable instructions that, when executed by the processor, cause the system at lea

Assignees

• Latta Stephen G
• Markovic Relja
• Tomlin Arthur Charles
• Snook Gregory N
• Microsoft Technology Licensing Llc

Inventors

• Latta Stephen G
• Markovic Relja
• Tomlin Arthur Charles
• Snook Gregory N

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06K9/00389

  Physics · mapped topic

• G06F3/033

  Pointing devices displaced or positioned by the user {, e.g. mice, trackballs, pens or joysticks}; Accessories therefor (digitisers characterised by the transducing means G06F3/041) · CPC title

• A63F2300/6045

  for mapping control signals received from the input arrangement into game commands · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title