In-vehicle gesture interactive spatial audio system

Patent metadata

Field	Value
Publication number	US-2015193197-A1
Application number	US-201414587602-A
Country	US
Kind code	A1
Filing date	Dec 31, 2014
Priority date	Jan 3, 2014
Publication date	Jul 9, 2015
Grant date	—

Abstract

One embodiment of the present invention sets forth a technique for modifying an audio parameter based on a gesture. The technique includes acquiring sensor data associated with a hand of a user and analyzing the sensor data to determine at least one hand position. The technique further includes determining, based on the at least one hand position, an interaction between a first virtual object that corresponds to an audio event and a second virtual object that corresponds to the hand of the user. The technique further includes, based on the interaction, modifying a spatial audio parameter associated with the audio event to generate a modified audio stream and causing the modified audio stream to be reproduced for output to the user.

First claim

What is claimed is:

1 . A method for modifying an audio parameter based on a gesture, the method comprising:
acquiring sensor data associated with a hand of a user; analyzing the sensor data to determine at least one hand position; determining, based on the at least one hand position, an interaction between a first virtual object that corresponds to an audio event and a second virtual object that corresponds to the hand of the user; based on the interaction, modifying a spatial audio parameter associated with the audio event to generate a modified audio stream; and causing the modified audio stream to be reproduced for output to the user.

2 . The method of claim 1 , wherein modifying the spatial audio parameter comprises moving the audio event from a first location within a sound space towards a second location within the sound space.

3 . The method of claim 2 , further comprising displaying a visual representation of the first virtual object, and, based on the interaction, moving the visual representation from a first display position associated with the first location within the sound space to a second display position associated with the second location within the sound space.

4 . The method of claim 3 , wherein the visual representation comprises a three-dimensional image, and the sensor data comprises coordinates associated with the hand of the user in three-dimensional space.

5 . The method of claim 4 , wherein determining the interaction comprises detecting, based on the coordinates associated with the hand of the user, that the second virtual object is proximate to the first virtual object.

6 . The method of claim 3 , wherein determining the interaction comprises detecting that the second virtual object is proximate to the first virtual object.

7 . The method of claim 6 , further comprising causing at least one of auditory feedback and haptic feedback to be generated in response to detecting that the second virtual object is proximate to the first virtual object.

8 . The method of claim 3 , wherein analyzing the sensor data comprises determining a first hand position at which a grasping gesture was initiated, and determining a second hand position at which the grasping gesture was terminated, wherein the first hand position is associated with the first location within the sound space and the second hand position is associated with the second location within the sound space.

9 . The method of claim 3 , wherein the audio event comprises an ambient audio source that spans a portion of the sound space, and the visual representation spans a portion of the display that corresponds to the portion of the sound space.

10 . The method of claim 1 , further comprising:
determining, based on the at least one hand position, a second interaction between the first virtual object and the second virtual object; based on the second interaction, moving the audio event from a first sound space associated with the user to a second sound space associated with a second user to generate a second modified audio stream; and causing the second modified audio stream to be reproduced within the second sound space.

11 . A system for modifying an audio parameter based on a gesture, comprising:
at least one sensor configured to acquire sensor data associated with a hand of a user; a processor coupled to the at least one sensor and configured to:
analyze the sensor data to determine at least one hand position;
determine, based on the at least one hand position, an interaction between a first virtual object that corresponds to an audio event and a second virtual object that corresponds to the hand of the user;
based on the interaction, modify a spatial audio parameter associated with the audio event to generate a modified audio stream; and
cause the modified audio stream to be reproduced for output to the user; and
a display coupled to the processor and configured to display a visual representation of the first virtual object.

12 . The system of claim 11 , wherein the processor is configured to modify the spatial audio parameter by moving the audio event from a first location within a sound space towards a second location within the sound space.

13 . The system of claim 12 , wherein the processor is further configured to, based on the interaction, cause the visual representation to move from a first position on the display associated with the first location within the sound space to a second position on the display associated with the second location within the sound space.

14 . The system of claim 11 , wherein the display comprises a volumetric display, and the visual representation comprises a three-dimensional image.

15 . The system of claim 14 , wherein the sensor data comprises first coordinates associated with the hand of the user in three-dimensional space, and the processor is configured to determine the interaction by determining that the first coordinates associated with the hand of the user are proximate to second coordinates associated with the second virtual object.

16 . The system of claim 14 , wherein the processor is configured to determine the interaction by determining that the hand of the user is performing a gesture proximate to the three-dimensional image.

17 . The system of claim 11 , wherein the at least one hand position comprises first hand positions associated with the user, and second hand positions associated with a second user, and the processor is further configured to:
determine, based on the second hand positions, a second interaction between a third virtual object that corresponds to a second audio event and a fourth virtual object that corresponds to a hand of the second user; based on the second interaction, moving the second audio event from a second sound space associated with the second user to a first sound space associated with the user.

18 . The system of claim 11 , wherein the processor is further configured to detect, based on the at least one hand position, that the second virtual object is proximate to the first virtual object and, in response, cause at least one of visual feedback, auditory feedback, and haptic feedback to be generated.

19 . The system of claim 11 , wherein the audio event comprises an ambient audio source, and the processor is configured to modify the spatial audio parameter by modifying a portion of a sound space that is spanned by the audio event.

20 . A non-transitory computer-readable storage medium including instructions that, when executed by a processor, cause the processor to modify an audio parameter based on a gesture, by performing the steps of:
acquiring sensor data associated with a hand of a user; analyzing the sensor data to determine at least one hand position; determining, based on the at least one hand position, an interaction between a first virtual object that corresponds to an audio event and a second virtual object that corresponds to the hand of the user; based on the interaction, resizing a portion of a sound space spanned by the audio event to generate a modified audio stream; and causing the modified audio stream to be reproduced for output to the user.

Assignees

• Harman Int Ind

Inventors

• Nahman Jaime Elliot
• Di Censo Davide
• Marti Stefan

Classifications

• H04R2499/13

  Acoustic transducers and sound field adaptation in vehicles · CPC title

• G06F3/162

  Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/016

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title