Adaptive audio construction

Patent metadata

Field	Value
Publication number	US-10728688-B2
Application number	US-201916424409-A
Country	US
Kind code	B2
Filing date	May 28, 2019
Priority date	Feb 3, 2015
Publication date	Jul 28, 2020
Grant date	Jul 28, 2020

Abstract

Described herein is a method for creating an object-based audio signal from an audio input, the audio input including one or more audio channels that are recorded to collectively define an audio scene. The one or more audio channels are captured from a respective one or more spatially separated microphones disposed in a stable spatial configuration. The method includes the steps of: a) receiving the audio input; b) performing spatial analysis on the one or more audio channels to identify one or more audio objects within the audio scene; c) determining contextual information relating to the one or more audio objects; d) defining respective audio streams including audio data relating to at least one of the identified one or more audio objects; and e) outputting an object-based audio signal including the audio streams and the contextual information.

First claim

We claim:

1. A method of creating an object-based audio signal from an audio input, the audio input including one or more audio channels that are recorded to collectively define an audio scene, the one or more audio channels being captured from a respective one or more spatially separated microphones disposed in a stable spatial configuration, the method comprising:
receiving, by a spatial audio analysis module, the audio input;
performing spatial analysis on the one or more audio channels to identify one or more audio objects within the audio scene;
determining contextual information relating to the one or more audio objects, wherein the contextual information includes spatial properties of the one or more audio objects;
defining respective audio streams including audio data relating to at least one of the identified one or more audio objects;
outputting, by the spatial audio analysis module, an object-based audio signal including the audio streams and the contextual information;
receiving, by an automated effects processing module, the object-based audio signal;
performing effects processing on one or more of the audio streams to generate a modified object-based audio signal, wherein the effects processing is based on the contextual information and on external context information that is external to the audio signal, and wherein the effects processing includes exaggerating spatial properties included in the contextual information; and
outputting, by the automated effects processing module, the modified object-based audio signal as an encoded signal.

2. A method according to claim 1 , further comprising:
receiving, by the spatial audio analysis module, the external context information, the external context information being relevant to the audio input.

3. A method according to claim 1 , wherein the spatial analysis is performed based on the external context information.

4. A method according to claim 1 , further comprising:
selectively manipulating one or more of the audio streams to modify spatial properties of associated audio objects of the one or more audio streams, the selective manipulating including biasing the spatial properties.

5. A method according to claim 4 , wherein the biasing is performed based at least in part on the contextual information.

6. A method according to claim 4 , wherein the biasing is performed based at least in part on the external context information.

7. A method according to claim 4 , wherein selectively manipulating one or more of the audio streams includes modifying a perceived direction of travel of an audio object within the audio scene.

8. A method according to claim 4 , wherein selectively manipulating one or more of the audio streams includes modifying a background and/or foreground audio scene component.

9. A method according to claim 4 , wherein selectively manipulating one or more of the audio streams includes assigning to an audio object a spatial trajectory through the audio scene.

10. A method according to claim 4 , wherein selectively manipulating one or more of the audio streams includes modifying a perceived velocity of an audio object through the audio scene.

11. A method according to claim 1 , wherein the external context information includes directional properties of the audio scene.

12. A method according to claim 11 , wherein the directional properties include a location of a microphone.

13. A method according to claim 1 , wherein the external context information includes control input from a user.

14. A method according to claim 1 , wherein the contextual information includes an object type.

15. A method according to claim 1 , wherein the spatial properties include one or more of size, shape, position, coherence, direction of travel, velocity or acceleration of an audio object relative to the spatial configuration.

16. A method according to claim 1 , wherein the audio objects include one or more of voice, ambient sounds, instruments and noise.

17. A method according to claim 1 , wherein the audio input includes a plurality of audio channels, and wherein the step of defining respective audio streams includes performing a beamforming technique on the plurality of audio channels.

18. A method according to claim 1 , wherein the audio input includes a plurality of audio channels, and wherein performing spatial audio analysis includes performing one or more of beamforming, audio event detection, level estimation, spatial clustering, spatial classification and temporal data analysis.

19. A computer-based system including a processor configured to perform the method according to claim 1 .

20. A non-transitory computer-readable medium storing instructions that are, when executed by one or more processors, operable to cause the one or more processors to perform the method according to claim 1 .

Assignees

• Dolby Laboratories Licensing Corp

Inventors

• Dickins Glenn N
• Cartwright Richard J

Classifications

• H04S7/00 Primary

  Indicating arrangements; Control arrangements, e.g. balance control · CPC title

• H04S2400/15

  Aspects of sound capture and related signal processing for recording or reproduction · CPC title

• H04R5/027

  Spatial or constructional arrangements of microphones, e.g. in dummy heads · CPC title

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title

• G10L19/008

  Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing · CPC title