Musical analysis platform

Patent metadata

Field	Value
Publication number	US-9804818-B2
Application number	US-201514871897-A
Country	US
Kind code	B2
Filing date	Sep 30, 2015
Priority date	Sep 30, 2015
Publication date	Oct 31, 2017
Grant date	Oct 31, 2017

Abstract

A platform or system is disclosed for performing musical analysis to detect musical properties in received live or pre-recorded audio data. The analysis can include a synchronous analysis for generating estimated one or more transitory musical properties and an asynchronous analysis for generating one or more aggregate musical properties which can be applied to the transitory musical properties to generate confirmed musical properties, which can be stored as metadata associated with an audio file. In some cases, live audio data can be received, recorded, dynamically analyzed to provide realtime metadata (e.g., to a display), then the realtime metadata can be analyzed to provide confirmed, updated, or validated metadata. In some cases, initial analysis (e.g., dynamic analysis) can determine chord estimates, usable in further analysis (e.g., offline analysis) to estimate a musical key, which can then be applied to the chord estimates to determine the most likely chord estimates and determine chord progressions.

First claim

What is claimed is:

1. A system, comprising:
one or more data processors; and
a non-transitory computer-readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform operations including:
receiving incoming audio data;
detecting one or more transients in the audio data;
splitting the audio data into a plurality of audio segments, wherein splitting the audio data includes using the one or more transients;
processing the plurality of audio segments by a frequency analyzer to identify one or more peak frequencies in one of the plurality of audio segments;
selecting a target frequency of an audio segment by a target frequency selector from the one or more peak frequencies, wherein the target frequency corresponds to a reference note having a reference frequency;
processing the audio segment to determine a tuning offset including calculating a difference in frequency between the target frequency and the reference frequency; and
applying the tuning offset using a frequency adjustor when the audio data includes untuned audio data, wherein applying the tuning offset includes applying the tuning offset to the plurality of audio segments.

2. The system of claim 1 , wherein selecting the target frequency includes:
determining correlation scores between each of the one or more peak frequencies and each of one or more reference notes, wherein determining a correlation score includes determining a likelihood that a peak frequency corresponds to a reference note; and
choosing the target frequency form the one or more peak frequencies as having the highest of the correlation scores.

3. The system of claim 1 , wherein selecting the target frequency includes:
choosing a target frequency from the one or more peak frequencies;
determining correlation scores between the target frequency and each of a plurality of reference notes, wherein determining a correlation score includes determining a likelihood that the target frequency corresponds to a reference note; and
selecting the reference note from the plurality of references notes as having the highest of the correlation scores.

4. The system of claim 1 , wherein selecting the target frequency further includes identifying that the target frequency falls within a preset frequency window, and wherein applying the tuning offset includes adjusting one or more benchmark frequency values when the one or more benchmark frequency values falls within the preset frequency window.

5. The system of claim 4 , wherein the operations further include:
determining an additional tuning offset for an additional target frequency that falls within an additional preset frequency window, wherein the additional preset frequency window corresponds to notes that do not fall within the preset frequency window, and wherein applying the tuning offset includes adjusting an additional one or more benchmark frequency values when the additional one or more benchmark frequency values falls within the additional preset frequency window.

6. The system of claim 1 , wherein the operations further include:
selecting an additional frequency from one or more additional peak frequencies of the one of the plurality of audio segments or another of the plurality of audio segments, wherein the selected additional frequency corresponds to an additional reference note;
determining an additional tuning offset by calculating a difference in frequency between the selected additional frequency and an additional reference frequency of the additional reference note; and
updating the tuning offset using the additional tuning offset.

7. The system of claim 1 , wherein applying the tuning offset includes adjusting one or more benchmark frequency values, and wherein the operations further include:
using the one or more adjusted benchmark frequency values to estimate one or more semitones present in the audio data; and
determining a musical property of the audio data using the one or more estimated semitones.

8. The system of claim 7 , wherein the musical property is a chord transcription of the audio data corresponding to one or more chords present across a duration of the audio data, wherein determining the musical property includes determining a set of chord candidates having one or more notes matching the one or more estimated semitones.

9. The system of claim 7 , wherein determining the musical property includes determining a confidence score using the one or more semitones and wherein the confidence score is influenced by a difference between the one or more semitones and one or more expected values.

10. The system of claim 1 , wherein applying the tuning offset including adjusting one or more benchmark frequency values or adjusting one or more frequencies of the plurality of audio segments.

11. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause a data processing apparatus to perform operations including:
receiving incoming audio data;
detecting one or more transients in the audio data;
splitting the audio data into a plurality of audio segments, wherein splitting the audio data includes using the one or more transients;
processing the plurality of audio segments by a frequency analyzer to identify one or more peak frequencies in one of the plurality of audio segments;
selecting a target frequency of an audio segment by a target frequency selector from the one or more peak frequencies, wherein the target frequency corresponds to a reference note having a reference frequency;
processing the audio segment to determine a tuning offset including calculating a difference in frequency between the target frequency and the reference frequency; and
applying the tuning offset using a frequency adjustor when the audio data includes untuned audio data, wherein applying the tuning offset includes applying the tuning offset to the plurality of audio segments.

12. The computer-program product of claim 11 , wherein selecting the target frequency includes:
determining correlation scores between each of the one or more peak frequencies and each of one or more reference notes, wherein determining a correlation score includes determining a likelihood that a peak frequency corresponds to a reference note; and
choosing the target frequency form the one or more peak frequencies as having the highest of the correlation scores.

13. The computer-program product of claim 11 , wherein selecting the target frequency includes:
choosing a target frequency from the one or more peak frequencies;
determining correlation scores between the target frequency and each of a plurality of reference notes, wherein determining a correlation score includes determining a likelihood that the target frequency corresponds to a reference note; and
selecting the reference note from the plurality of references notes as having the highest of the correlation scores.

14. The computer-program product of claim 11 , wherein selecting the target frequency further includes identifying that the target frequency falls within a preset frequency window, and wherein applying the tuning offset includes adjusting one or more benchmark frequency values when the one or more benchmark frequency values falls within the preset frequency window.

15. The computer-program product of claim 14 , wherein the operations further include:
determining an additional tuning offset for an additional target frequency that falls within an additional preset frequency window, wherein the additional preset

Assignees

• Apple Inc

Inventors

• Gehring Steffen
• Fournier Pierre

Classifications

• G10G1/00 Primary

  Means for the representation of music · CPC title

• G06F3/165 Primary

  Management of the audio stream, e.g. setting of volume, audio stream path · CPC title

• G10H2210/081

  for automatic key or tonality recognition, e.g. using musical rules or a knowledge base · CPC title

• G10H1/383

  Chord detection and/or recognition, e.g. for correction, or automatic bass generation · CPC title

• G10H2210/076

  for extraction of timing, tempo; Beat detection · CPC title