Systems and methods for composition of audio content from multi-object audio

Patent metadata

Field	Value
Publication number	US-10901685-B2
Application number	US-201916440199-A
Country	US
Kind code	B2
Filing date	Jun 13, 2019
Priority date	Feb 21, 2018
Publication date	Jan 26, 2021
Grant date	Jan 26, 2021

Abstract

Embodiments are related to processing of one or more input audio feeds for generation of a target audio stream that includes at least one object of interest to a listener. In some embodiments, the target audio stream may exclusively or primarily include the sound of the object of interest to the listener, without including other persons. This allows a listener to focus on an object of his or her interest and not necessarily have to listen to the performances of other objects in the input audio feed. Some embodiments contemplate multiple audio feeds and/or with multiple objects of interest.

First claim

What is claimed is:

1. A method for composition of audio content comprising:
receiving a first audio feed and a second audio feed, each feed including one or more objects distributed in multiple frames;
generating a fingerprint of at least a portion of the first audio feed and a fingerprint of at least a portion of the second audio feed;
retrieving, from a database, a fingerprint of a first object of interest and a fingerprint of a second object of interest;
comparing the fingerprint of at least the portion of the first audio feed with the fingerprint of the first object of interest to detect first matched frames that include the fingerprint of the first object of interest;
comparing the fingerprint of at least the portion of the second audio feed with the fingerprint of the second object of interest to detect second matched frames that include the fingerprint of the second object of interest; and
generating a target audio stream having the first object of interest and the second object of interest by compositing the first matched frames and the second matched frames.

2. The method of claim 1 , wherein comparing the fingerprint of at least the portion of the first audio feed with the fingerprint of the first object of interest fingerprint of the second object of interest are associated with partial elimination of objects that do not correspond to the first and second objects of interest, from the target audio stream.

3. The method of claim 1 , wherein comparing the fingerprint of at least the portion of the first audio feed with the fingerprint of the first object of interest and comparing the fingerprint of at least the portion of the second audio feed with the fingerprint of the second object of interest are associated with complete elimination of objects that do not correspond to the first and second objects of interest, from the target audio stream.

4. The method of claim 1 further comprising:
creating a first timeline of the first audio feed and a second timeline of the second audio feed; and
annotating the first and second timelines of the audio feeds at positions corresponding to positions of the first and second matched frames, respectively.

5. The method of claim 4 , wherein the annotating allows playing of the target audio stream at the positions of the first and second matched frames.

6. The method of claim 4 , wherein the annotating allows selective seeking the positions of the first and second matched frames, without generation of the target audio stream.

7. A non-transitory computer-readable storage medium storing instructions configured for composition of audio content to perform a method comprising:
receiving a first audio feed and a second audio feed, each feed including one or more objects distributed in multiple frames;
generating a fingerprint of at least a portion of the first audio feed and a fingerprint of at least a portion of the second audio feed;
retrieving, from a database, a fingerprint of a first object of interest and a fingerprint of a second object of interest;
comparing the fingerprint of at least the portion of the first audio feed with the fingerprint of the first object of interest to detect first matched frames that include the fingerprint of the first object of interest;
comparing the fingerprint of at least the portion of the second audio feed with the fingerprint of the second object of interest to detect second matched frames that include the fingerprint of the second object of interest; and
compositing the first matched frames and the second matched frames to generate a target audio stream having the first object of interest and the second object of interest.

8. The computer-readable storage medium of claim 7 , wherein comparing the fingerprint of at least the portion of the first audio feed with the fingerprint of the first object of interest and comparing the fingerprint of at least the portion of the second audio feed with the fingerprint of the second object of interest are associated with partial elimination of objects that do not correspond to the first and second objects of interest, from the target audio stream.

9. The computer-readable storage medium of claim 7 , wherein the matching causes, from the target audio stream, complete elimination of objects that do not correspond to the first and second objects of interest to the listener.

10. The computer-readable storage medium of claim 7 , the method further comprising:
creating a first timeline of the first audio feed and a second timeline of the second audio feed;
annotating the first and second timelines of the audio feeds at positions corresponding to positions of the first and second matched frames, respectively.

11. An apparatus for composition of audio content comprising:
a memory; and
one or more processors electronically coupled to the memory and configured for:
receiving a first audio feed and a second audio feed, each feed including one or more objects distributed in multiple frames;
generating a fingerprint of at least a portion of the first audio feed and a fingerprint of at least a portion of the second audio feed;
retrieving, from a database, a fingerprint of a first object of interest and a fingerprint of a second object of interest;
comparing the fingerprint of at least the portion of the first audio feed with the fingerprint of the first object of interest to detect first matched frames that include the fingerprint of the first object of interest;
comparing the fingerprint of at least the portion of the second audio feed with the fingerprint of the second object of interest to detect second matched frames that include the fingerprint of the second object of interest; and
compositing the first matched frames and the second matched frames to generate a target audio stream having the first object of interest and the second object of interest.

12. The apparatus of claim 11 , wherein comparing the fingerprint of at least the portion of the first audio feed with the fingerprint of the first object of interest and comparing the fingerprint of at least the portion of the second audio feed with the fingerprint of the second object of interest are associated with partial elimination of objects that do not correspond to the first and second objects of interest, from the target audio stream.

13. The apparatus of claim 11 , wherein comparing the fingerprint of at least the portion of the first audio feed with the fingerprint of the first object of interest and comparing the fingerprint of at least the portion of the second audio feed with the fingerprint of the second object of interest are associated with complete elimination of objects that do not correspond to the first and second objects of interest, from the target audio stream.

14. The apparatus of claim 11 , wherein the one or more processors are further configured for:
creating a first timeline of the first audio feed and a second timeline of the second audio feed; and
annotating the first and second timelines of the audio feeds at positions corresponding to positions of the first and second matched frames, respectively.

Assignees

• Sling Media Pvt Ltd

Inventors

• Raikar Yatish Jayant Naik
• Rasool Mohammed
• Pallapothu Trinadha Harish Babu

Classifications

• G06F3/165 Primary

  Management of the audio stream, e.g. setting of volume, audio stream path · CPC title

• G10L25/51 Primary

  for comparison or discrimination · CPC title