Interactive audio content generation, delivery, playback and sharing

Patent metadata

Field	Value
Publication number	US-9411882-B2
Application number	US-201414334802-A
Country	US
Kind code	B2
Filing date	Jul 18, 2014
Priority date	Jul 22, 2013
Publication date	Aug 9, 2016
Grant date	Aug 9, 2016

Abstract

Control data templates are generated independent of a plurality of audio elements based on user input. The user input relates to parameter values and control inputs for operations. In response to receiving audio elements after the control data templates are generated, audio objects are generated to store audio sample data representing the audio elements. Control data is generated based on the parameter values and the control inputs for the operations in the control data templates. The control data specifies the operations to be performed while rendering the audio objects. The control data is then stored separately from the audio sample data in the audio objects. The audio objects can be communicated to downstream recipient devices for rendering and/or remixing.

First claim

What is claimed is:

1. A method, comprising:
generating, based on user input, independent of a plurality of audio elements, one or more control data templates, the user input relating to a plurality of parameter values and a plurality of control inputs for a plurality of operations, the one or more control data templates comprising the plurality of parameter values and the plurality of control inputs for the plurality of operations;
wherein the one or more control data templates are generated, before the one or more control data templates are applied to the plurality of audio elements to create a plurality of audio objects storing audio sample data representing the plurality of audio elements;
after generating the one or more control data templates, receiving the plurality of audio elements; and
in response to receiving the plurality of audio elements,
creating the plurality of audio objects to store the audio sample data representing the plurality of audio elements;
generating control data based on the plurality of parameter values and the plurality of control inputs for the plurality of operations, the control data specifying the plurality of operations to be performed while rendering the plurality of audio objects; and
storing the control data separately from the audio sample data in the plurality of audio objects;

wherein the method is performed by one or more computing devices.

2. The method as recited in claim 1 , wherein the plurality of audio elements is generated based at least in part on one of: premixing audio tracks or decoding previously mixed audio data.

3. The method as recited in claim 1 , further comprising outputting the plurality of audio objects to a downstream media device in one of media data bitstreams, audio signals, audiovisual signals, media data files, media data containers, or media data structures.

4. The method as recited in claim 1 , wherein the control data is at least partially editable without causing changes to the audio sample data by a recipient device to create a remix different from an original mix represented by the plurality of audio objects.

5. The method as recited in claim 1 , wherein at least one operation in the plurality of operations is synchronized to one or more occurrences of a control input in the plurality of control inputs when the plurality of audio objects is rendered.

6. The method as recited in claim 1 , wherein the plurality of operations comprises a sequence of operations each of which is caused to be performed at an occurrence of a respective trigger in the sequence of triggers.

7. The method as recited in claim 1 , wherein a time duration of an operation in the plurality of operations is scaled to fill a time duration of one or more of particular beats, particular measures, or particular time periods, in audio content represented by the plurality of audio objects.

8. The method as recited in claim 1 , further comprising storing the plurality of control data templates into a data repository as a set of individually accessible control data template libraries.

9. The method as recited in claim 1 , wherein the plurality of operations comprises one or more non-audio operations.

10. A method, comprising:
decoding a plurality of audio objects into audio sample data representing a plurality of audio elements and control data specifying a plurality of operations to be performed at occurrences of respective control inputs in a plurality of control inputs while rendering the plurality of audio objects;
wherein the plurality of audio objects are created by an upstream encoder by applying one or more control data templates to the plurality of audio elements;
wherein the one or more control data templates are generated, before the one or more control data templates are applied to the plurality of audio elements to create the plurality of audio objects storing the audio sample data representing the plurality of audio elements; and
while rendering the plurality of audio objects, performing the plurality of operations at occurrences of a plurality of control input specified in the control data decoded from the plurality of audio objects;
wherein the method is performed by one or more computing devices.

11. The method as recited in claim 10 , wherein the control data specifies one or more of audio-related operations or non-audio operations.

12. The method as recited in claim 10 , wherein the control data is at least partially editable without causing changes to the audio sample data by a recipient device to create a remix different from an original mix represented by the plurality of audio objects.

13. The method as recited in claim 10 , wherein at least one operation in the plurality of operations is synchronized to one or more occurrences of a control input in the plurality of control inputs while rendering the plurality of audio objects.

14. The method as recited in claim 10 , wherein the plurality of operations comprises a sequence of operations each of which is caused to be performed at an occurrence of a respective trigger in the sequence of triggers.

15. The method as recited in claim 10 , wherein a time duration of an operation in the plurality of operations is scaled to fill a time duration of one or more of particular beats, particular measures, or particular time periods, in audio content represented by the plurality of audio objects.

16. The method as recited in claim 10 , wherein the plurality of operations comprises one or more non-audio operations.

17. A method, comprising:
decoding a plurality of audio objects into audio sample data representing a plurality of audio elements and control data specifying a plurality of operations to be performed at occurrences of respective control inputs in a plurality of control inputs while rendering the plurality of audio objects;
wherein the plurality of audio objects are created by an upstream encoder by applying one or more control data templates to the plurality of audio elements;
wherein the one or more control data templates are generated, before the one or more control data templates are applied to the plurality of audio elements to create the plurality of audio objects storing the audio sample data representing the plurality of audio elements; and
generating one or more modifications to the control data so that a portion of the control data related to at least one operation in the plurality of operations is changed;
wherein the method is performed by one or more computing devices.

18. The method as recited in claim 17 , further comprising, while rendering the plurality of audio objects, performing the at least one operation in the plurality of operations differently than originally specified in the control data as decoded without the one or more modifications.

19. The method as recited in claim 17 , further comprising, while rendering the plurality of audio objects, performing the at least one operation in the plurality of operations at occurrences of a different control input than a control input originally specified in the control data as decoded without the one or more modifications.

20. The method as recited in claim 17 , further comprising saving the one or more modifications to the control data without the audio sample data in a separately deliverable one of media data streams, media data files, media data containers, or media data structures.

Assignees

• Dolby Laboratories Licensing Corp

Inventors

• Cory Gabriel Grant
• Kustka Joel Robert
• Vernon Stephen D
• Esten Ian Eric
• Poirier Sophia Genung
• Nurmukhanov Dossym

Classifications

• G10H1/0025

  Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece (automatically producing a series of tones G10H1/26) · CPC title

• G10H2210/131

  Morphing, i.e. transformation of a musical piece into a new different one, e.g. remix · CPC title

• H04S7/30 Primary

  Control circuits for electronic adaptation of the sound field · CPC title

• G06F17/3074 Primary

  Physics · mapped topic

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title