Methods and systems for generating and interactively rendering object based audio

Patent metadata

Field	Value
Publication number	US-9805727-B2
Application number	US-201414781562-A
Country	US
Kind code	B2
Filing date	Apr 3, 2014
Priority date	Apr 3, 2013
Publication date	Oct 31, 2017
Grant date	Oct 31, 2017

Abstract

Methods for generating an object based audio program, renderable in a personalizable manner, and including a bed of speaker channels renderable in the absence of selection of other program content (e.g., to provide a default full range audio experience). Other embodiments include steps of delivering, decoding, and/or rendering such a program. Rendering of content of the bed, or of a selected mix of other content of the program, may provide an immersive experience. The program may include multiple object channels (e.g., object channels indicative of user-selectable and user-configurable objects), the bed of speaker channels, and other speaker channels. Another aspect is an audio processing unit (e.g., encoder or decoder) configured to perform, or which includes a buffer memory which stores at least one frame (or other segment) of an object based audio program (or bitstream thereof) generated in accordance with, any embodiment of the method.

First claim

What is claimed is:

1. A method for generating an object based audio program indicative of audio content including first non-ambient content, second non-ambient content different than the first non-ambient content, and third content different than the first non-ambient content and the second non-ambient content, said method including steps of:
determining a set of object channels consisting of N object channels, where a first subset of the set of object channels is indicative of the first non-ambient content, the first subset consists of M object channels of the set of object channels, each of N and M is an integer greater than zero, and M is equal to or less than N;
determining a bed of speaker channels indicative of a default mix of audio content, where an object based speaker channel subset consisting of M of the speaker channels of the bed is indicative of the second non-ambient content or a mix of the second non-ambient content and at least some other audio content of the audio content of the default mix;
determining a set of M replacement speaker channels, where each replacement speaker channel in the set of M replacement speaker channels is indicative of none or some, but not all, of the content of a corresponding speaker channel of the object based speaker channel subset;
generating metadata indicative of at least one selectable predetermined alternative mix of content of at least one of the object channels and content of predetermined ones of the speaker channels of the bed and/or the replacement speaker channels, where the metadata includes rendering parameters for each said alternative mix, and at least one said alternative mix is a replacement mix indicative of at least some of the audio content of the bed and the first non-ambient content, but not the second non-ambient content; and
generating the object based audio program to include the bed of speaker channels, the set of M replacement speaker channels, the set of object channels, and the metadata, such that the bed of speaker channels is renderable without use of the metadata to provide sound perceivable as the default mix, and the replacement mix is renderable, in response to at least some of the metadata, to provide sound perceivable as a mix including said at least some of the audio content of the bed and the first non-ambient content but not the second non-ambient content.

2. The method of claim 1 , wherein at least some of the metadata is selectable content metadata indicative of a set of selectable predetermined mixes of audio content of the program, and including a predetermined set of rendering parameters of each of the predetermined mixes.

3. The method of claim 1 , wherein at least some of the metadata is indicative of a mix graph, the mix graph is indicative of selectable mixes of the speaker channels of the bed, the replacement speaker channels and the object channels, the object based audio program is an encoded bitstream comprising frames, and each of the frames of the encoded bitstream includes metadata indicative of the mix graph.

4. A method of rendering audio content determined by an object based audio program, wherein the program is indicative of a bed of speaker channels, a set of M replacement speaker channels, a set of object channels, and metadata, wherein the set of object channels consists of N object channels, a first subset of the set of object channels is indicative of first non-ambient content, the first subset consists of M object channels of the set of object channels, each of N and M is an integer greater than zero, and M is equal to or less than N,
the bed of speaker channels is indicative of a default mix of audio content, including second non-ambient content different than the first non-ambient content, where an object based speaker channel subset consisting of M of the speaker channels of the bed is indicative of the second non-ambient content or a mix of the second non-ambient content and at least some other audio content of the audio content of the default mix,
each replacement speaker channel in the set of M replacement speaker channels is indicative of none or some, but not all, of the content of a corresponding speaker channel of the object based speaker channel subset, and
the metadata is indicative of at least one selectable predetermined alternative mix of content of at least one of the object channels and content of predetermined ones of the speaker channels of the bed and/or the replacement speaker channels, where the metadata includes rendering parameters for each said alternative mix, and at least one said alternative mix is a replacement mix including at least some of the audio content of the bed and the first non-ambient content, but not the second non-ambient content, said method including steps of:
(a) providing the object based audio program to an audio processing unit; and
(b) in the audio processing unit, parsing the bed of speaker channels and rendering the default mix in response to the bed of speaker channels without use of the metadata.

5. The method of claim 4 wherein the audio processing unit is configured to parse the object channels and the metadata of the program, said method also including the step of:
(c) in the audio processing unit, rendering the replacement mix using at least some of the metadata, including by selecting, and mixing content of, the first subset of the set of object channels and at least one said replacement speaker channel in response to at least some of the metadata.

6. The method of claim 5 , wherein step (c) includes steps of:
(d) in response to said at least some of the metadata, selecting the first subset of the set of object channels, selecting at least one speaker channel of the bed of speaker channels other than a speaker channel in the object based speaker channel subset, and selecting said at least one said replacement speaker channel; and
(e) mixing content of the first subset of the set of object channels, and of each speaker channel selected in step (d), thereby determining the replacement mix.

7. The method of claim 4 , wherein at least some of the metadata is indicative of a mix graph, the mix graph is indicative of selectable mixes of the speaker channels of the bed, the replacement speaker channels and the object channels, the object based audio program is an encoded bitstream comprising frames, and each of the frames of the encoded bitstream includes metadata indicative of the mix graph.

8. A system for generating an object based audio program indicative of audio content including first non-ambient content, second non-ambient content different than the first non-ambient content, and third content different than the first non-ambient content and the second non-ambient content, said system including:
a first subsystem configured to determine:
a set of object channels consisting of N object channels, where a first subset of the set of object channels is indicative of the first non-ambient content, the first subset consists of M object channels of the set of object channels, each of N and M is an integer greater than zero, and M is equal to or less than N,
a bed of speaker channels indicative of a default mix of audio content, where an object based speaker channel subset consisting of M of the speaker channels of the bed is indicative of the second non-ambient content or a mix of the second non-ambient content and at least some other audio content of the audio content of the default mix, and
a set of M replacement speaker channels, where each replacement speaker channel in the set of M replacement speaker channels is indicative of none or some, but not all, of the content of a corresponding speaker channel of the object based speaker channel subset, wherein the first subsystem is also configu

Assignees

• Dolby Laboratories Licensing Corp
• Dolby Int Ab

Inventors

• Mehta Sripal S
• Ziegler Thomas
• Baker Giles
• Riedmiller Jeffrey
• Saungsomboon Prinyar

Classifications

• H04S2420/03

  Application of parametric coding in stereophonic audio systems · CPC title

• H04S2400/13

  Aspects of volume control, not necessarily automatic, in stereophonic sound systems · CPC title

• H04S7/30

  Control circuits for electronic adaptation of the sound field · CPC title

• H04S2400/01

  Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved · CPC title

• G10L19/20

  using sound class specific coding, hybrid encoders or object based coding · CPC title