Method and apparatus for generating digital score file of song, and storage medium

Patent metadata

Field	Value
Publication number	US-10923089-B2
Application number	US-201816059109-A
Country	US
Kind code	B2
Filing date	Aug 9, 2018
Priority date	Sep 9, 2016
Publication date	Feb 16, 2021
Grant date	Feb 16, 2021

Abstract

A method and an information processing apparatus to generate a digital score file of a song are described. The information processing apparatus includes processing circuitry. The processing circuitry is configured to obtain a candidate audio file satisfying a first condition from audio files of unaccompanied singing of the song without instrumental accompaniment. The processing circuitry is configured to divide the candidate audio file into valid audio segments based on timing information of the song, and extract pieces of music note information from the valid audio segments. Each of the pieces of music note information includes at least one data set of a music note in the song. The data set includes an onset time, a duration, and a music note value of the music note. The processing circuitry is configured to generate the digital score file based on the pieces of music note information.

First claim

What is claimed is:

1. A method for generating a digital score file that records musical notation of a song, comprising:
obtaining, by processing circuitry of an information processing apparatus, audio files of unaccompanied singing of the song without instrumental accompaniment, the audio files being uploaded to the information processing apparatus;
obtaining reply information corresponding to the audio files;
determining quality levels of the audio files based on the reply information;
identifying a candidate audio file from the audio files, a corresponding quality level of the candidate audio file from the determined quality levels being greater than a threshold; and
after the candidate audio file is identified, generating the digital score file that records the musical notation of the song according to the candidate audio file, the generating the digital score file including:
dividing, by the processing circuitry of the information processing apparatus, the candidate audio file into valid audio segments based on timing information of the song;
extracting pieces of music note information from the valid audio segments, each of the pieces of music note information including at least one data set of a music note in the song, the data set including an onset time, a duration, and a music note value of the music note; and
generating the digital score file based on the pieces of music note information.

2. The method according to claim 1 , wherein the dividing the candidate audio file into the valid audio segments comprises:
obtaining a lyric file corresponding to the song, the lyric file including pieces of line timing information corresponding to lyric lines of the song; and
dividing the candidate audio file into the valid audio segments based on the pieces of line timing information.

3. A method for generating a digital score file that records musical notation of a song, comprising:
obtaining, by processing circuitry of an information processing apparatus, a candidate audio file satisfying a first condition from audio files of unaccompanied singing of the song without instrumental accompaniment;
dividing, by the processing circuitry of the information processing apparatus, the candidate audio file into valid audio segments based on timing information of the song;
extracting pieces of music note information from the valid audio segments, each of the pieces of music note information including at least one data set of a music note in the song, the data set including an onset time, a duration, and a music note value of the music note; and
generating the digital score file based on the pieces of music note information,
wherein the extracting the pieces of music note information from the valid audio segments comprises:
for each of the valid audio segments,
extracting fundamental frequency information from the respective valid audio segment, the fundamental frequency information including at least one fundamental frequency value associated with the respective valid audio segment, and
when the at least one fundamental frequency value is not zero, adding a play duration of the respective valid audio segment to a segment duration; and

when the segment duration exceeds a preset duration,
processing the respective valid audio segment to obtain at least one fundamental frequency band,
obtaining the at least one fundamental frequency value from the at least one fundamental frequency band,
implementing music notation processing on the at least one fundamental frequency value to obtain at least one music note value, and
merging the music note values for the valid audio segments to obtain the pieces of music note information associated with the valid audio segments.

4. The method according to claim 3 , wherein
the processing the respective valid audio segment includes:
obtaining a length corresponding to the respective valid audio segment; and
implementing a first filtering process on the respective valid audio segment to obtain a first filtered segment when the length is less than a first preset value and implementing a second filtering process on the respective valid audio segment to obtain a second filtered segment when the length is greater than or equal to the first preset value; and

the obtaining the at least one fundamental frequency value includes:
obtaining the at least one fundamental frequency value based on the first filtered segment when the length is less than the first preset value and obtaining the at least one fundamental frequency value based on the second filtered segment when the length is greater than or equal to the first preset value.

5. The method according to claim 4 , further comprising:
when the length is less than a second preset value and the at least one fundamental frequency value is zero, setting the at least one fundamental frequency value to be a preset fundamental frequency value; and
obtaining the preset fundamental frequency value.

6. The method according to claim 3 , further comprising:
obtaining the fundamental frequency values associated with adjacent pairs of the valid audio segments;
calculating a difference between the fundamental frequency values; and
setting the fundamental frequency values to zero when the difference satisfies a second condition.

7. The method according to claim 3 , wherein the merging the music note values comprises:
merging adjacent music note values that are identical to generate updated music note values of updated music notes,
generating, based on the updated music note values, multiple data sets of the updated music notes, the multiple data sets corresponding to the valid audio segments; and
combining the multiple data sets of the updated music notes into a first sequence indicating the pieces of music note information.

8. The method according to claim 7 , wherein the generating the multiple data sets of the updated music notes further comprises:
obtaining respective onset times and durations associated with the updated music notes that correspond to the valid audio segments, and
generating the multiple data sets corresponding to the valid audio segments based on the respective onset times, the durations, and the updated music note values of the updated music notes.

9. The method according to claim 7 , wherein
the first sequence includes a first data set of the multiple data sets and a second data set of the multiple data sets, the first data set including a first onset time and a first duration, the second data set including a second onset time and a second duration; and
the method further includes:
when the first onset time is equal to a sum of the second onset time and the second duration, merging the first data set and the second data set to obtain a second sequence, the first data set and the second data set forming a third data set in the second sequence;
when the first onset time is not equal to the sum of the second onset time and the second duration and when an onset time of a fourth data set of the multiple data sets is equal to a sum of the first onset time and the first duration, merging the fourth data set and the first data set to obtain the second sequence, the fourth data set and the first data set forming a fifth data set in the second sequence;
when the first onset time is not equal to the sum of the second onset time and the second duration and when the onset time of the fourth data set is not equal to the sum of the first onset time and the first duration, deleting the first data set from the first sequence to obtain the second sequence; and
outputting the data sets in the second sequence in a preset format to obtain the digital score file.

10. The method according to claim 9 , wh

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Zhao Weifeng

Classifications

• G10G3/04 Primary

  using electrical means · CPC title

• G10H1/361

  Recording/reproducing of accompaniment for use with an external source, e.g. karaoke systems · CPC title

• G10H2210/071

  for rhythm pattern analysis or rhythm style recognition · CPC title

• G10H2210/086

  for transcription of raw audio or music data to a displayed or printed staff representation or to displayable MIDI-like note-oriented data, e.g. in pianoroll format · CPC title

• G10H2230/015

  PDA [personal digital assistant] or palmtop computing devices used for musical purposes, e.g. portable music players, tablet computers, e-readers or smart phones in which mobile telephony functions need not be used · CPC title