Active trigger poses

Patent metadata

Field	Value
Publication number	US-10092827-B2
Application number	US-201615184781-A
Country	US
Kind code	B2
Filing date	Jun 16, 2016
Priority date	Jun 16, 2016
Publication date	Oct 9, 2018
Grant date	Oct 9, 2018

Abstract

Systems, methods and articles of manufacture for controlling electronic devices in an interactive gaming environment. Embodiments detect a first interactive game is playing within the physical environment using one or more electronic devices. User movement is monitored using at least one sensor device within the physical environment and user speech is monitored using one or more microphone sensor devices within the physical environment. Upon determining that the user movement matches a predefined series of user actions and that the user speech matches a corresponding predefined speech pattern, embodiments determine a gameplay action corresponding to both the predefined series of user actions and the predefined speech pattern and transmit an instruction, to at least one of the one or more electronic devices within the physical environment, instructing the electronic device to perform the determined gameplay action.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
detecting a first interactive game is playing within a physical environment using one or more electronic devices;
monitoring user movement using at least one sensor device within the physical environment;
monitoring user speech using one or more microphone sensor devices within the physical environment;
upon determining that the user movement matches a predefined series of user actions and that the user speech matches a corresponding predefined speech pattern:
determining a gameplay action corresponding to both the predefined series of user actions and the predefined speech pattern and based on a current context of a story being played in the first interactive game, and wherein the first interactive game includes a plurality of distinct contexts;
determining an in-game ability to perform within the first interactive game based on the current context of the story, wherein the predefined series of user actions and the predefined speech pattern correspond to a respective distinct in-game ability in each of the plurality of distinct contexts;
performing the determined in-game ability within the first interactive game and by operation of one or more computer processors; and
transmitting an instruction, to at least one of the one or more electronic devices within the physical environment, instructing the at least one electronic device to perform the determined gameplay action.

2. The computer-implemented method of claim 1 , wherein the at least one sensor device within the physical environment comprises one or more camera devices, and further comprising:
analyzing a plurality of frames captured by the one or more camera devices in order to identify a respective plurality of pixels representing a user in each of the plurality of frames; and
determining the user movement based on changes in the plurality of pixels across the plurality of frames.

3. The computer-implemented method of claim 1 , wherein the at least one sensor device within the physical environment comprises one or more electromyography sensors, and further comprising:
analyzing an electromyogram collected by the one or more electromyography sensors to determine muscle activations made by the user.

4. The computer-implemented method of claim 3 , wherein determining that the user movement matches the predefined series of user actions further comprises:
determining that the determined muscle activations made by the user match a predefined pattern of muscle activations.

5. The computer-implemented method of claim 1 , wherein the at least one sensor device within the physical environment comprises an inertial movement unit, and further comprising:
analyzing data collected by the inertial movement unit to determine a measure of exerted force and an angular rate of the user movement.

6. The computer-implemented method of claim 5 , wherein determining that the user movement matches the predefined series of user actions further comprises:
determining that the measure of exerted force and angular rate match a predefined pattern corresponding to the predefined series of user actions.

7. The computer-implemented method of claim 1 , wherein the user movement comprises at least one of a twisting motion and a spinning motion, and wherein the user speech comprises at least one of a spoken phrase and a song.

8. The computer-implemented method of claim 1 , wherein transmitting an instruction, to at least one of the one or more electronic devices within the physical environment, instructing the at least one electronic device to perform the determined gameplay action further comprises:
transmitting an instruction to an augmented reality device within the physical environment, specifying to render a plurality of frames depicting a performance of the in-game ability.

9. The computer-implemented method of claim 8 , further comprising:
transmitting an instruction to an interactive electronic device within the physical environment, instructing the interactive electronic device to perform one or more actions in response to the performance of the in-game ability.

10. The computer-implemented method of claim 9 , wherein the one or more actions comprise at least one of (i) outputting a sound effect, (ii) outputting a lighting effect using one or more illumination devices, (iii) initiating a haptic feedback mechanism.

11. The computer-implemented method of claim 8 , further comprising:
upon receiving, by the augmented reality device, the instruction specifying to render the plurality of frames depicting the performance of the in-game ability:
determining one or more predefined virtual objects corresponding to the in-game ability;
determining a virtual position for the one or more predefined virtual objects, based on a determined position of a user action within one or more frames captured using a camera sensor of the augmented reality device;
determining a size and orientation for the one or more predefined virtual objects, relative to a size of the user action within the one or more frames; and
rendering a plurality of frames for display on a display device of the augmented reality device, in which the one or more predefined virtual objects are positioned according to the determined virtual position and in which the one or more predefined virtual objects are depicted at the determined size and orientation.

12. The computer-implemented method of claim 1 , wherein determining the gameplay action and transmitting the instruction are not performed unless both the user movement matches the predefined series of user actions and the user speech matches the corresponding predefined speech pattern.

13. A non-transitory computer-readable medium, containing computer program code that, when executed, performs an operation comprising:
detecting a first interactive game is playing within a physical environment using one or more electronic devices;
monitoring user movement using at least one sensor device within the physical environment;
monitoring user speech using one or more microphone sensor devices within the physical environment;
upon determining that the user movement matches a predefined series of user actions and that the user speech matches a corresponding predefined speech pattern:
determining a gameplay action corresponding to both the predefined series of user actions and the predefined speech pattern and based on a current context of a story being played in the first interactive game, and wherein the first interactive game includes a plurality of distinct contexts;
determining an in-game ability to perform within the first interactive game based on the current context of the story, wherein the predefined series of user actions and the predefined speech pattern correspond to a respective distinct in-game ability in each of the plurality of distinct contexts;
performing the determined in-game ability within the first interactive game and by operation of one or more computer processors when executing the computer program code; and
transmitting an instruction, to at least one of the one or more electronic devices within the physical environment, instructing the at least one electronic device to perform the determined gameplay action.

14. The non-transitory computer-readable medium of claim 13 , wherein the at least one sensor device within the physical environment comprises one or more camera devices, and further comprising:
analyzing a plurality of frames captured by the one or more camera devices in order to identify a respective plurality of pixels representing a user in each of the plurality of frames; an

Assignees

• Disney Entpr Inc

Inventors

• Goslin Michael P
• Olson Blade A

Classifications

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• A63F13/20 Primary

  Input arrangements for video game devices · CPC title

• A63F13/21

  characterised by their sensors, purposes or types · CPC title