Computer system for producing audio content for realizing customized being-there and method thereof

Patent metadata

Field	Value
Publication number	US-11930349-B2
Application number	US-202117534823-A
Country	US
Kind code	B2
Filing date	Nov 24, 2021
Priority date	Nov 24, 2020
Publication date	Mar 12, 2024
Grant date	Mar 12, 2024

Abstract

Provided are a computer system for producing audio content for realizing a user-customized being-there and a method thereof. The computer system may be configured to generate audio files based on respective audio signals that are respectively generated from a plurality of objects at a venue, set spatial features at the venue for the objects, respectively, using a production tool, and generate metadata for the audio files based on the spatial features. An electronic device may realize a being-there at the venue by rendering the audio files based on the spatial features in the metadata. That is, a user of the electronic device may feel a user-customized being-there as if the user directly listens to audio signals generated from corresponding objects at a venue in which the objects are provided.

First claim

What is claimed is:

1. A method by a computer system, the method comprising:
generating audio files based on respective audio signals, the audio signals having been respectively generated from a plurality of objects at a venue;
setting spatial features at the venue for the objects, respectively, using a production tool; and
generating metadata for the audio files based on the spatial features,
wherein the setting comprises,
outputting a graphic interface,
setting the spatial features for the objects, respectively, based on at least one input through the graphic interface, and
storing the spatial features in association with the objects, respectively.

2. The method of claim 1 , wherein the metadata includes at least one of position information about each of the objects, group information representing a position combination of at least two objects among the objects, and environment information about the venue.

3. The method of claim 1 , wherein each of the objects includes one of a musical instrument, an instrument player, a vocalist, a talker, a speaker, and a background.

4. The method of claim 1 , wherein
the graphic interface includes,
a first area for displaying the objects at the venue, and
a second area displayed on a same screen as that of the first area and for setting a position of an object selected from the first area, and

the setting the spatial features comprises setting each of the spatial features based on the position.

5. The method of claim 4 , wherein
the graphic interface further includes a third area displayed on the same screen as that of the first area and for fine-tuning an audio effect for the object selected from the first area, and
the setting comprises setting each of the spatial features based on the position and the audio effect.

6. The method of claim 4 , wherein
the graphic interface further includes at least one of
a third area for displaying at least one venue, and
a fourth area displayed on a same screen as that of the third area and for fine-tuning an audio effect related to a select venue selected from the third area, and

the setting comprises setting each of the spatial features based on the audio effect.

7. The method of claim 6 , wherein the third area is displayed on a same area as that of the first area or displayed on an area different from that of the first area.

8. The method of claim 1 , further comprising at least one of:
rendering the audio files based on the metadata; and
storing the audio files and the metadata together.

9. A method by a computer system, the method comprising:
generating audio files based on respective audio signals, the audio signals having been respectively generated from a plurality of objects at a venue;
setting spatial features at the venue for the objects, respectively, using a production tool;
generating metadata for the audio files based on the spatial features; and
transmitting the audio files and the metadata together,
wherein the transmitting comprises composing the audio files and the metadata as a pulse code modulation (PCM) audio signal and transmitting the same, and
the metadata is embedded in a metadata track of the PCM audio signal, synchronized with the audio files based on a frame size of an audio codec to be used for encoding the audio files and the metadata, and is included as a plurality of sets in a single frame.

10. A non-transitory computer-readable record medium storing a program, which when executed by at least one processor included in a computer system, causes the computer system to perform the method of claim 1 .

11. A computer system comprising:
a memory; and
a processor configured to connect to the memory and to execute at least one instruction stored in the memory to cause the computer system to,
generate audio files based on audio signals, respectively, the audio signals having been generated from a plurality of objects at a venue, respectively,
set spatial features at the venue for the objects, respectively, using a production tool, and
generate metadata for the audio files based on the spatial features,
wherein the processor is further configured to cause the computer system to, output a graphic interface to set the spatial features for the objects,
respectively, based on at least one input through the graphic interface, and store the spatial features in association with the objects, respectively.

12. The computer system of claim 11 , wherein the metadata includes at least one of position information about each of the objects, group information representing a position combination of at least two objects among the objects, and environment information about the venue.

13. The computer system of claim 11 , wherein each of the objects includes one of a musical instrument, an instrument player, a vocalist, a talker, a speaker, and a background.

14. The computer system of claim 11 , wherein
the graphic interface includes,
a first area for displaying the objects at the venue, and
a second area displayed on a same screen as that of the first area and for setting a position of an object selected from the first area, and

the processor is further configured to cause the computer system to set each of the spatial features based on the position.

15. The computer system of claim 14 , wherein
the graphic interface further includes a third area displayed on the same screen as that of the first area and for fine-tuning an audio effect for the object selected from the first area, and
the processor is configured to cause the computer system to set each of the spatial features based on the position and the audio effect.

16. The computer system of claim 14 , wherein
the graphic interface further includes at least one of,
a third area for displaying at least one venue, and
a fourth area displayed on a same screen as that of the third area and for fine-tuning an audio effect related to a select venue selected from the fourth area, and

the processor is further configured to cause the computer system to set each of the spatial features based on the audio effect.

17. The computer system of claim 11 , wherein the processor is further configured to cause the computer system to render the audio files based on the metadata, store the audio files and the metadata together, or transmit the audio files and the metadata together.

18. The computer system of claim 17 , wherein the processor is further configured to
cause the computer system to compose the audio files and the metadata as a pulse code modulation (PCM) audio signal and to transmit the same, and
the metadata is embedded in a metadata track of the PCM audio signal, synchronized with the audio files based on a frame size of an audio codec to be used for encoding the audio files and the metadata, and is included as a plurality of sets in a single frame.

Assignees

• Naver Corp

Inventors

• Kim Dae Hwang
• Kim Jung Sik
• Kim Dong Hwan
• Lee Ted
• Seo Jeonghun
• Oh Jiwon

Classifications

• H04S7/30 Primary

  Control circuits for electronic adaptation of the sound field · CPC title

• G06F3/04847

  Interaction techniques to control parameter settings, e.g. interaction with sliders or dials · CPC title

• G06F3/165

  Management of the audio stream, e.g. setting of volume, audio stream path · CPC title

• G06F16/61

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F16/687

  using geographical or spatial information, e.g. location · CPC title