Musical analysis platform

Patent metadata

Field	Value
Publication number	US-9852721-B2
Application number	US-201514871902-A
Country	US
Kind code	B2
Filing date	Sep 30, 2015
Priority date	Sep 30, 2015
Publication date	Dec 26, 2017
Grant date	Dec 26, 2017

Abstract

A platform or system is disclosed for performing musical analysis to detect musical properties in received live or pre-recorded audio data. The analysis can include a synchronous analysis for generating estimated one or more transitory musical properties and an asynchronous analysis for generating one or more aggregate musical properties which can be applied to the transitory musical properties to generate confirmed musical properties, which can be stored as metadata associated with an audio file. In some cases, live audio data can be received, recorded, dynamically analyzed to provide realtime metadata (e.g., to a display), then the realtime metadata can be analyzed to provide confirmed, updated, or validated metadata. In some cases, initial analysis (e.g., dynamic analysis) can determine chord estimates, usable in further analysis (e.g., offline analysis) to estimate a musical key, which can then be applied to the chord estimates to determine the most likely chord estimates and determine chord progressions.

First claim

What is claimed is:

1. A system, comprising:
one or more data processors; and
a non-transitory computer-readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform operations including:
synchronous receiving and processing incoming audio data, wherein the incoming audio data is processed by a transitory analyzer;
generating one or more estimated transitory musical properties corresponding to the incoming audio data;
storing the one or more estimated transitory musical properties;
storing the incoming audio data as an audio file;
asynchronously processing the stored one or more estimated transitory musical properties, wherein the stored one or more estimated transitory musical properties are processed by a musical property analyzer;
generating an aggregate musical property corresponding to the audio file, wherein the aggregate musical property is generated using the stored one or more estimated transitory musical properties;
generating one or more confirmed musical properties corresponding to the audio file, wherein the one or more confirmed musical properties are generated using the aggregate musical property and the stored one or more estimated transitory musical properties; and
storing the one or more confirmed musical properties as musical metadata associated with the audio file.

2. The system of claim 1 , wherein the aggregate musical property is a musical key estimate of the audio file for the duration.

3. The system of claim 2 , wherein the one or more transitory musical properties includes chord estimate data representing one or more possible chords present in the audio data at a moment in a duration of the audio data, and wherein using the aggregate musical property to generate the one or more confirmed musical properties includes using the musical key estimate to select a best fit chord from the one or more possible chords of the chord estimate data for the moment.

4. The system of claim 1 , wherein the one or more transitory musical properties includes chord estimate data representing one or more possible chords present in the audio data at a moment in a duration of the audio data, wherein the aggregate musical property is an estimated chord progression, and wherein using the aggregate musical property to generate the one or more confirmed musical properties includes using the estimated chord progression to select a best fit chord from the one or more possible chords of the chord estimate data for the moment.

5. The system of claim 1 , wherein the one or more confirmed musical properties includes a chord transcription representing one or more chords present in the audio data for a duration of the audio data.

6. The system of claim 5 , wherein the one or more transitory musical properties includes a monophonic transcription of the audio file for the duration of the audio data.

7. The system of claim 5 , wherein the one or more transitory musical properties includes rhythmic data corresponding to a timing of beats in the audio data, and wherein using the aggregate musical property to generate the one of more confirmed musical properties includes selecting the best fit chord using the rhythmic data.

8. The system of claim 1 , wherein generating the one or more estimated transitory musical properties includes separating an audio segment out of the audio data, determining that the audio segment is polyphonic, identifying a plurality of notes in the audio data, and generating a set of one or more possible chords for the audio segment using the plurality of notes.

9. The system of claim 8 , wherein generating the set of one or more possible chords includes generating a set of one or more possible primary chords and generating a set of one or more possible chord extensions.

10. The system of claim 1 , wherein the operations further include processing buffered audio data by the transitory analyzer to generate one or more additional estimated transitory musical properties corresponding to the buffered audio data, wherein the one or more confirmed musical properties are generated using the aggregate musical property, the one or more estimated transitory musical properties, and the one or more additional estimated transitory musical properties.

11. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause a data processing apparatus to perform operations including:
synchronous receiving and processing incoming audio data, wherein the incoming audio data is processed by a transitory analyzer;
generating one or more estimated transitory musical properties corresponding to the incoming audio data;
storing the one or more estimated transitory musical properties;
storing the incoming audio data as an audio file;
asynchronously processing the stored one or more estimated transitory musical properties, wherein the stored one or more estimated transitory musical properties are processed by a musical property analyzer;
generating an aggregate musical property corresponding to the audio file, wherein the aggregate musical property is generated using the stored one or more estimated transitory musical properties;
generating one or more confirmed musical properties corresponding to the audio file, wherein the one or more confirmed musical properties are generated using the aggregate musical property and the stored one or more estimated transitory musical properties; and
storing the one or more confirmed musical properties as musical metadata associated with the audio file.

12. The computer-program product of claim 11 , wherein the aggregate musical property is a musical key estimate of the audio file for the duration.

13. The computer-program product of claim 12 , wherein the one or more transitory musical properties includes chord estimate data representing one or more possible chords present in the audio data at a moment in a duration of the audio data, and wherein using the aggregate musical property to generate the one or more confirmed musical properties includes using the musical key estimate to select a best fit chord from the one or more possible chords of the chord estimate data for the moment.

14. The computer-program product of claim 11 , wherein the one or more transitory musical properties includes chord estimate data representing one or more possible chords present in the audio data at a moment in a duration of the audio data, wherein the aggregate musical property is an estimated chord progression, and wherein using the aggregate musical property to generate the one or more confirmed musical properties includes using the estimated chord progression to select a best fit chord from the one or more possible chords of the chord estimate data for the moment.

15. The computer-program product of claim 11 , wherein the one or more confirmed musical properties includes a chord transcription representing one or more chords present in the audio data for a duration of the audio data.

16. The computer-program product of claim 15 , wherein the one or more transitory musical properties includes a monophonic transcription of the audio file for the duration of the audio data.

17. The computer-program product of claim 15 , wherein the one or more transitory musical properties includes rhythmic data corresponding to a timing of beats in the audio data, and wherein using the aggregate musical property to generate the one of more confirmed musical properties includes selecting the best fit chord using the rhythmic

Assignees

• Apple Inc

Inventors

• Kozielski Christoph
• Fournier Pierre

Classifications

• G10H1/0025 Primary

  Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece (automatically producing a series of tones G10H1/26) · CPC title

• G10H2210/086

  for transcription of raw audio or music data to a displayed or printed staff representation or to displayable MIDI-like note-oriented data, e.g. in pianoroll format · CPC title

• G10H2210/081

  for automatic key or tonality recognition, e.g. using musical rules or a knowledge base · CPC title

• G10H1/0008 Primary

  Associated control or indicating means · CPC title

• G10H2210/076

  for extraction of timing, tempo; Beat detection · CPC title