Searching for Music

Patent metadata

Field	Value
Publication number	US-2021294840-A1
Application number	US-202016823538-A
Country	US
Kind code	A1
Filing date	Mar 19, 2020
Priority date	Mar 19, 2020
Publication date	Sep 23, 2021
Grant date	—

Abstract

In implementations of searching for music, a music search system can receive a music search request that includes a music file including music content. The music search system can also receive a selected musical attribute from a plurality of musical attributes. The music search system includes a music search application that can generate musical features of the music content, where a respective one or more of the musical features correspond to a respective one of the musical attributes. The music search application can then compare the musical features that correspond to the selected musical attribute to audio features of audio files, and determine similar audio files to the music file based on the comparison of the musical features to the audio features of the audio files.

First claim

What is claimed is:

1 . In a digital medium environment for searching for music, a method implemented by a computing device, the method comprising:
receiving a music search request that includes a music file including music content; receiving a selected musical attribute from a plurality of musical attributes; generating musical features of the music content, a respective one or more of the musical features corresponding to a respective one of the plurality of musical attributes; comparing the respective one or more musical features corresponding to the selected musical attribute to audio features of a plurality of audio files; and determining similar audio files to the music file based on the comparing of the respective one or more musical features to the audio features of the plurality of audio files.

2 . The method as described in claim 1 , wherein the plurality of musical attributes include two or more of a genre attribute, a mood attribute, an instrument attribute, and a tempo attribute.

3 . The method as described in claim 1 , further comprising providing audio content of the similar audio files for playback by the computing device.

4 . The method as described in claim 1 , wherein the generating the musical features of the music content includes generating a feature vector that includes a representation of each of the musical features, the feature vector partitioned into independent subspaces that each represent a respective musical attribute of the plurality of musical attributes.

5 . The method as described in claim 4 , wherein the comparing of the respective one or more musical features to the audio features includes determining distances of the feature vector to feature vectors representing the audio features over one of the independent subspaces that corresponds to the selected musical attribute.

6 . The method as described in claim 5 , further comprising:
comparing the distances to a distance similarity threshold; and determining the similar audio files to the music file based on one or more of the distances that are less than the distance similarity threshold.

7 . The method as described in claim 5 , further comprising:
ranking the plurality of audio files based on the distances; and determining the similar audio files to the music file based on the ranking.

8 . The method as described in claim 1 , wherein the generating the musical features of the music content includes:
generating a plurality of additional features from respective portions of the music file; and averaging the plurality of additional features to generate the musical features.

9 . The method as described in claim 1 , further comprising:
receiving an additional music file including additional music content as part of the music search request; generating additional musical features of the additional music content, a respective one or more of the additional musical features also corresponding to a respective one of the plurality of musical attributes; combining the musical features of the music content with the additional musical features of the additional music content into averaged musical features; comparing the averaged musical features corresponding to the selected musical attribute to the audio features of the plurality of audio files; and determining the similar audio files to the music file based on the comparing of the averaged musical features to the audio features.

10 . The method as described in claim 1 , further comprising:
generating a representation of frequencies of the music content in the music file; and generating, with a neural network, a feature vector that includes the musical features of the music content based on the representation of frequencies, wherein the neural network has coefficients determined from triplets of the audio files, each triplet including an anchor file having anchor content, a positive file having positive content that is similar to the anchor content according to at least one of the plurality of musical attributes, and a negative file having negative content that is dissimilar to the anchor content according to the at least one of the plurality of musical attributes.

11 . A music search system implemented by a computing device for searching for music in a digital medium environment, the music search system comprising:
a memory to maintain prototype feature vectors having elements that represent a respective one of a plurality of musical attributes, each of the prototype feature vectors corresponding to respective text-based tags that indicate one or more of the musical attributes, and each of the prototype feature vectors formed by averaging feature vectors that represent music files tagged with a respective one of the text-based tags; a processor system to implement a music search application at least partially in hardware of the computing device to:
receive a selected text-based tag as part of a music search request, the selected text-based tag indicating a musical attribute of the plurality of musical attributes;
determine a matching prototype feature vector from the prototype feature vectors based on the selected text-based tag;
compare the elements of the matching prototype feature vector that correspond to the musical attribute indicated by the selected text-based tag to additional feature vectors of a plurality of audio files; and
determine, based on the compare, information of one or more of the plurality of audio files to display on the computing device.

12 . The music search system as described in claim 11 , wherein the music search application is implemented to receive the selected text-based tag as part of the music search request without receiving a music file as part of the music search request.

13 . The music search system as described in claim 11 , wherein the music search application is implemented to:
receive a music file as part of the music search request; receive a degree of desired dissimilarity for an additional musical attribute of the plurality of musical attributes; generate an additional feature vector from the music file; determine a projection of the additional feature vector onto the matching prototype feature vector; determine an orthogonalized feature vector by subtracting an amount of the projection from the matching prototype feature vector, the amount based on the degree of desired dissimilarity; compare the orthogonalized feature vector to the additional feature vectors of the plurality of audio files; and determine, based on the compare the orthogonalized feature vector to the additional feature vectors, information of at least one of the plurality of audio files to display on the computing device.

14 . The music search system as described in claim 11 , wherein the music search application is implemented to:
receive a music file as part of the music search request; generate an additional feature vector from the music file; combine the matching prototype feature vector and the additional feature vector to form an averaged feature vector; compare the averaged feature vector to the additional feature vectors of the plurality of audio files; and determine, based on the compare the averaged feature vector to the additional feature vectors, information of at least one of the plurality of audio files to display on the computing device.

15 . The music search system as described in claim 11 , wherein the music search application is implemented to compare the elements of the matching prototype feature vector to the additional feature vectors by determining norms of differences between the m

Assignees

• Adobe Inc

Inventors

• Lee Jongpil
• Bryan Nicholas J
• Salamon Justin J
• Jin Zeyu

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G10L25/51

  for comparison or discrimination · CPC title