Coordinating and mixing vocals captured from geographically distributed performers

Patent metadata

Field	Value
Publication number	US-11074923-B2
Application number	US-201916550769-A
Country	US
Kind code	B2
Filing date	Aug 26, 2019
Priority date	Apr 12, 2010
Publication date	Jul 27, 2021
Grant date	Jul 27, 2021

Abstract

Despite many practical limitations imposed by mobile device platforms and application execution environments, vocal musical performances may be captured and continuously pitch-corrected for mixing and rendering with backing tracks in ways that create compelling user experiences. Based on the techniques described herein, even mere amateurs are encouraged to share with friends and family or to collaborate and contribute vocal performances as part of virtual “glee clubs.” In some implementations, these interactions are facilitated through social network- and/or eMail-mediated sharing of performances and invitations to join in a group performance. Using uploaded vocals captured at clients such as a mobile device, a content server (or service) can mediate such virtual glee clubs by manipulating and mixing the uploaded vocal performances of multiple contributing vocalists.

First claim

What is claimed is:

1. A method of coordinating a group performance, wherein the coordinated group performance includes vocal contributions captured at respective geographically-distributed portable computing devices, the method comprising:
using a first one of the geographically-distributed portable computing devices for vocal performance capture, the portable computing device having a display, a microphone interface and a communications interface;
responsive to a user selection, receiving via the communications interface (i) a backing track including a vocal performance of at least one other vocalist captured at a remote one of the geographically-distributed portable computing devices and (ii) a vocal score temporally synchronizable with the backing track and with lyrics, wherein the vocal score encodes a sequence of notes for at least a portion of a vocal melody;
at the first portable computing device, rendering the backing track and concurrently presenting corresponding portions of the lyrics on the display in temporal correspondence therewith;
at the first portable computing device, and against the rendered backing track, capturing and pitch correcting at least a portion of a vocal performance of the user in accord with the vocal score; and
streaming, as the coordinated group performance, a mix of the user's captured vocal performance with the vocal performance of the at least one other vocalist captured at the remote portable computing device, wherein metadata associated with the streamed mix includes geocoded location of the respective vocal performance captures for display in association with an audiovisual rendering of the coordinated group performance.

2. The method of claim 1 , further comprising:
capturing, in association with the captures of vocal performances of the user and the at least one other vocalist, the respective geocoded locations of the first portable computing device and the remote portable computing device for display to a listener in association with the audiovisual rendering of the coordinated group performance.

3. The method of claim 1 , further comprising:
displaying visually on a listener's portable computing device, in association with the audiovisual rendering of the coordinated group performance, the respective geocoded locations.

4. The method of claim 3 ,
wherein the visual displays of the respective geocoded locations are presented on a globe at positions visually suggestive to the listener of particular locations of the respective vocal performances.

5. The method of claim 4 ,
wherein the globe is presented as an interactive visual display animation rotatable by user gesture on a touchscreen of the listener's portable computing device and including the vocal performance locations presented on the interactive visual display animation at positions consistent with the respective geocoded locations.

6. The method of claim 3 ,
wherein the listener and the listener's portable computing device are distinct from the user and the at least one other vocalist and their respective portable computing devices.

7. The method of claim 3 ,
wherein the listener portable computing device is that of the user or the at least one other vocalist.

8. The method of claim 1 , further comprising:
displaying visually on a portable computing device, in association with the audiovisual rendering of the coordinated group performance, respective geocoded locations for listeners thereto.

9. The method of claim 8 ,
wherein the visual displays of the respective geocoded locations include indicia of listener feedback presented on a globe at positions visually suggestive of a particular location of the respective listener.

10. The method of claim 9 ,
wherein the globe is presented as an interactive visual display animation rotatable by user gesture on a touchscreen of a displaying portable computing device and including the listener feedback indicia presented on the interactive visual display animation at positions consistent with the respective geocoded locations or geotags for the listeners.

11. The method of claim 1 ,
wherein the pitch correcting at the portable computing device pitch shifts at least some portions of the user's captured vocal performance to harmony.

12. The method of claim 1 ,
wherein the vocal score codes at least a Part A or Part B duet portion of an overall vocal score.

13. The method of claim 1 ,
wherein the vocal score codes at least one or the other of melody and harmony portions of an overall vocal score.

14. The method of claim 1 , further comprising:
at the first portable computing device, capturing still images and/or video in association with the vocal capture for inclusion in the audiovisual rendering of the coordinated group performance.

15. The method of claim 1 , embodied, at least in part, as instructions executable on respective of the portable computing devices to perform the vocal performance capture, receiving of the backing track and vocal score and the pitch correcting.

16. The method of claim 15 , further embodied as instructions executable on respective of the portable computing devices to perform the visual displaying in association with the audiovisual rendering of the coordinated group performance, the respective geocoded locations.

17. The method of claim 15 , further embodied as respective subsets of the instructions executable on respective of the portable computing devices and/or content server platforms to stream the coordinated group performance with the associated geocoded locations.

18. A system comprising:
geographically-distributed portable computing devices for capture of media content for a coordinated group performance that includes at least vocal performance contributions captured at the respective portable computing devices, the portable computing devices each having a display, a microphone interface and a communications interface;
the communications interface of at least a first one of the portable computing devices configured to receive (i) a backing track including a vocal performance of at least one other vocalist captured at a different, remote one of the geographically-distributed portable computing devices and (ii) a vocal score temporally synchronizable with the received backing track and with lyrics, wherein the vocal score encodes a sequence of notes for at least a portion of a vocal melody;
the first portable computing device further configured (i) to render the received backing track and to concurrently present, in temporal correspondence therewith, corresponding portions of the lyrics on the display thereof and (ii) to, against the rendered backing track, capture and pitch correct at least a portion of a vocal performance of the user in accord with the vocal score; and
a content service platform configured to stream, as the coordinated group performance, a mix of the user's captured vocal performance with the vocal performance of the at least one other vocalist captured at the remote portable computing device, wherein metadata associated with the streamed mix includes geocoded location of the respective vocal performance captures for display in association with an audiovisual rendering of the coordinated group performance.

19. The system of claim 18 ,
wherein the first and remote ones of the portable computing devices are further configured to capture, in association with the captures of vocal performances of the user and the at least one other vocalist, respective geocoded locations of the respective portable computing devices fo

Assignees

• Smule Inc

Inventors

• Cook Perry R
• Lazier Ari
• Lieber Tom
• Kirk Turner Evan

Classifications

• G10L21/00

  Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility (G10L19/00 takes precedence) · CPC title

• G10L21/013 Primary

  Adapting to target pitch · CPC title

• G10H1/366

  with means for modifying or correcting the external signal, e.g. pitch correction, reverberation, changing a singer's voice · CPC title

• H04S7/30

  Control circuits for electronic adaptation of the sound field · CPC title

• G10H1/0058

  Transmission between separate instruments or between individual components of a musical system (G10H1/0083 takes precedence) · CPC title