Synthesizing audio of a venue

Patent metadata

Field	Value
Publication number	US-11202162-B2
Application number	US-201916678811-A
Country	US
Kind code	B2
Filing date	Nov 8, 2019
Priority date	Oct 18, 2019
Publication date	Dec 14, 2021
Grant date	Dec 14, 2021

Abstract

Disclosed herein are system, method, and computer program product embodiments for replicating a remote venue in a local venue. An embodiment operates by receiving original audio. Thereafter, the original stream of audio is modified to produce modified audio based on an audio profile unique to a remote venue smaller than the local venue. The audio profile comprises a virtual representation of the remote venue including (i) a virtual audio source corresponding to the local venue's audio source and configured to produce the original audio and (ii) a virtual reverberation point corresponding to a real-world location in the remote venue and the portion of the local venue. As such, the modified audio is determined based on receipt of the original audio from the reverberation point. Thus, after correlating the modified audio to the local venue's audio source, the modified audio is provided by the audio source to the portion of the local venue.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving, by a computing device, an original stream of audio at a local venue having an audio source configured to provide audio to a portion of the local venue;
modifying, by the computing device, the original stream of audio to produce a modified stream of audio based on an audio profile unique to a remote venue smaller than the local venue;
correlating, by the computing device, the modified stream of audio to the audio source of the local venue; and
presenting, by the computing device, the modified stream of audio on the audio source to the portion of the local venue,
wherein:
the audio profile comprises a virtual representation of the remote venue,
the virtual representation comprises (i) a virtual audio source corresponding to the audio source of the local venue and configured to produce the original stream of audio and (ii) a first virtual reverberation point corresponding to a real-world location in the remote venue and the portion of the local venue, and
the modified stream of audio is determined based on receipt of the original stream of audio from the first virtual reverberation point.

2. The computer-implemented method of claim 1 , further comprising:
modifying, by the computing device, the original stream of audio to produce a second modified stream of audio based on the audio profile;
correlating, by the computing device, the modified stream of audio to a second audio source of the local venue configured to provide audio to a second portion of local venue; and
presenting, by the computing device, the modified stream of audio on the second audio source to the second portion of the local venue,
wherein:
the virtual representation further comprises (i) a second virtual audio source corresponding to the second audio source of the local venue and configured to produce the original stream of audio and (ii) a second virtual reverberation point corresponding to a second real-world location in the remote venue, and
the modified stream of audio is further determined based on receipt of the original stream of audio from the second virtual reverberation point.

3. The computer-implemented method of claim 1 , wherein the first virtual reverberation point comprises a delay and a frequency-dependent level variation such that the modified stream of audio comprises the delay and the frequency-dependent level variation.

4. The computer-implemented method of claim 1 , wherein:
the audio source of the local venue corresponds to a predetermined location in the local venue, and
the virtual representation of the remote venue further comprises a predetermined virtual location based on the predetermined location in the local venue.

5. The computer-implemented method of claim 4 , wherein the original stream of audio is from a prerecorded audio source.

6. The computer-implemented method of claim 4 , wherein the original stream of audio is from a live audio source.

7. The computer-implemented method of claim 4 , further comprising:
identifying, by the computing device, a location of the audio source in the local venue;
wherein:
the virtual audio source is at a location in the virtual representation of the remote venue corresponding to the location of the live audio source in the local venue, and
the modified source of audio is based on the location of the virtual audio source in the virtual representation of the remote venue.

8. The computer-implemented method of claim 7 , further comprising:
tracking, by the computing device, a movement of the audio source in the local venue,
wherein:
the virtual audio source is moved to a different location in the virtual representation based on the movement of the audio source in the local venue, and
the modified stream of audio is further modified based on the different location of the virtual audio source in the virtual representation.

9. The computer-implemented method of claim 4 , further comprising:
determining, by the computing device, a degree of rotation of the audio source in the local venue,
wherein:
the virtual audio source is provided with a virtual degree of rotation relative to the degree of rotation of the audio source, and
the modified stream of audio is further modified based on the virtual degree of rotation of the first virtual audio source.

10. The computer-implemented method of claim 4 , further comprising:
identifying, by the computing device, a stream of images comprising the audio source in the local venue; and
determining, by the computing device, a degree of focus of a video camera recording the audio source in the stream of images,
wherein the modified stream of audio is further modified based on the degree of focus on the audio source in the local venue.

11. The computer-implemented method of claim 1 , wherein the audio profile of the remote venue includes an environmental audio present in or unique to the replicated venue, and further comprising:
mixing, by the computing device, the modified stream of audio with the environmental audio.

12. The computer-implemented method of claim 11 , wherein the environmental audio is provided at a frequency lower than the modified stream of audio.

13. The computer-implemented method of claim 1 , further comprising:
receiving, by the computing device, a stream of images corresponding to the original stream of audio; and
correlating, by the computing device, the stream of images to the modified stream of audio such that the stream of images is presented in sync with the modified stream of audio.

14. The computer-implemented method of claim 1 , further comprising:
sending, by the computing device, the modified stream of audio to a user device,
wherein the user device is configured to permit an authorized user to preview the modified stream of audio.

15. The computer-implemented method of claim 14 , wherein the user device permits the authorized user to edit the modified stream of audio.

16. The computer-implemented method of claim 1 , wherein the local venue is a dome.

17. The computer-implemented method of claim 1 , wherein the remote venue is an indoor venue.

18. The computer-implemented method of claim 1 , wherein the remote venue is an outdoor venue.

19. A system, comprising:
a memory; and
a processor coupled to the memory and configured to perform operations comprising:
receiving an original stream of audio at a local venue having an audio source configured to provide audio to a portion of the local venue;
modifying the original stream of audio to produce a modified stream of audio based on an audio profile unique to a remote venue smaller than the local venue;
correlating the modified stream of audio to the audio source of the local venue;
presenting the modified stream of audio on the audio source to the portion of the local venue,
wherein:
the audio profile comprises a virtual representation of the remote venue,
the virtual representation comprises (i) a virtual audio source corresponding to the audio source of the local venue and configured to produce the original stream of audio and (ii) a virtual reverberation point corresponding to a real-world location in the remote venue and the portion of the local venue, and
the modified stream of audio is determined based on receipt of the original stream of audio from the virtual reverberation point.

20. A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the a

Assignees

• Msg Entertainment Group Llc

Inventors

• Elby Stuart
• Hockman Erik

Classifications

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title

• H04S3/008

  in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02) · CPC title

• H04S7/307

  Frequency adjustment, e.g. tone control (H04S7/301 takes precedence) · CPC title

• H04S2400/01

  Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved · CPC title

• H04S7/306

  For headphones · CPC title