Techniques for providing non-verbal speech recognition in an immersive playtime environment

Patent metadata

Field	Value
Publication number	US-9855497-B2
Application number	US-201514601083-A
Country	US
Kind code	B2
Filing date	Jan 20, 2015
Priority date	Jan 20, 2015
Publication date	Jan 2, 2018
Grant date	Jan 2, 2018

Abstract

An immersive play environment platform including techniques describing recognizing non-verbal vocalization gestures from a user is disclosed. A headset device receives audio input from a user. The headset device transmits the audio input to a controller device. The controller device evaluates characteristics of the audio input (e.g., spectral features over a period of time) to determine whether the audio input corresponds to a predefined non-verbal vocalization, such as a humming noise, shouting noise, etc. The controller device may perform an action in response to detecting such non-verbal vocalizations, such as engaging a play object (e.g., an action figure, an action disc) in the play environment.

First claim

What is claimed is:

1. An immersive play experience platform providing an interactive environment, comprising:
an audio device configured to receive audio input; and
a controller device configured to perform an operation for recognizing non-verbal vocalizations, the operation comprising:
receiving audio input transmitted by the audio device,
determining that the audio input matches spectral features of one or more predefined non-verbal vocalizations characterizing gameplay within the interactive environment, and
causing an action corresponding to the one or more predefined non-verbal vocalizations to be performed on an interactive gameplay object within the interactive environment.

2. The platform of claim 1 , further comprising:
a physical device configured to receive instructions from the controller device to perform the corresponding action; and
wherein the interactive gameplay object is connected with the physical device.

3. The platform of claim 2 , wherein performing the corresponding action comprises:
causing the interactive gameplay object to disconnect from the physical device.

4. The platform of claim 1 , wherein the predefined non-verbal vocalizations include at least one of a humming gesture or a shouting gesture.

5. The platform of claim 1 , further comprising:
a sensor device having an accelerometer and a gyroscope, the sensor device configured to receive position and orientation input and to transmit the position and orientation input to the controller device.

6. The platform of claim 5 , wherein the one or more predefined non-verbal vocalizations are associated with a choreographed sequence having one or more gestures corresponding to specified position and orientation input transmitted from the sensor device.

7. The platform of claim 1 , wherein the predefined non-verbal vocalizations are further determined based on amplitude and timing thresholds measured in the audio input.

8. A method for recognizing non-verbal vocalizations, comprising:
receiving, by a controller device executing on an immersive play experience platform providing an interactive environment, audio input;
determining, by the controller device, that the audio input matches spectral features of one or more predefined non-verbal vocalizations characterizing gameplay within the interactive environment; and
causing an action corresponding to the one or more predefined non-verbal vocalizations to be performed on an interactive gameplay object within the interactive environment.

9. The method of claim 8 , further comprising, transmitting instructions to a physical device for performing the corresponding action.

10. The method of claim 9 , wherein performing the corresponding action comprises:
causing the interactive gameplay object to disconnect from the physical device.

11. The method of claim 8 , wherein the predefined non-verbal vocalizations include at least one of a humming gesture or a shouting gesture.

12. The method of claim 8 , further comprising, receiving position and orientation input.

13. The method of claim 12 , wherein the one or more predefined non-verbal vocalizations are associated with a choreographed sequence having one or more gestures corresponding to specified position and orientation input.

14. The method of claim 8 , wherein the predefined non-verbal vocalizations are further determined based on amplitude and timing thresholds measured in the audio input.

15. A non-transitory computer-readable storage medium storing instructions, which, when executed on a processor, perform an operation for recognizing non-verbal vocalizations, the operation comprising:
receiving, by a controller device executing on an immersive play experience platform providing an interactive environment, audio input;
determining, by the controller device, that the audio input matches spectral features of at least a first predefined non-verbal vocalization characterizing gameplay within the interactive environment;
determining that the matched audio input is received as a part of a predefined sequence of gestures including at least a first physical gesture and the first predefined non-verbal vocalization, and
causing an action corresponding to the predefined sequence of gestures to be performed on an interactive gameplay object.

16. The computer-readable storage medium of claim 15 , wherein the operation further comprises, transmitting instructions to a physical device for performing the corresponding action.

17. The computer-readable storage medium of claim 16 , wherein performing the corresponding action comprises:
causing, via the physical device, the interactive gameplay object to disconnect from the physical device.

18. The computer-readable storage medium of claim 15 , wherein the operation further comprises, receiving position and orientation input.

19. The computer-readable storage medium of claim 18 , wherein the at least the first physical gesture matches specified position and orientation input.

20. The computer-readable storage medium of claim 15 , wherein the at least the first predefined non-verbal vocalization is further determined based on amplitude and timing thresholds measured in the audio input.

Assignees

• Disney Entpr Inc

Inventors

• Goslin Michael P
• Haseltine Eric C
• Olson Joseph L

Classifications

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• G06F2203/0381

  Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer · CPC title

• A63F13/215 Primary

  comprising means for detecting acoustic signals, e.g. using a microphone · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title