Cognitive music engine using unsupervised learning

Patent metadata

Field	Value
Publication number	US-11562722-B2
Application number	US-201916443802-A
Country	US
Kind code	B2
Filing date	Jun 17, 2019
Priority date	Oct 12, 2015
Publication date	Jan 24, 2023
Grant date	Jan 24, 2023

Abstract

A method for generating a musical composition based on user input is described. A first set of musical characteristics from a first input musical piece is received as an input vector. The first set of musical characteristics is perturbed to create a perturbed input vector as input in a first set of nodes in a first visible layer of an unsupervised neural net. The unsupervised neural net comprised of a plurality of computing layers, each computing layer composed of a respective set of nodes. The unsupervised neural net is operated to calculate an output vector from a higher level hidden layer in the unsupervised neural net. The output vector is used to create an output musical piece.

First claim

Having described our invention, what we now claim is as follows:

1. A method for generating a musical composition, comprising:
receiving a first set of musical characteristics from a first input musical piece as an input vector and a user intent indicating a manner at least one of mood, length and genre in which an output piece is to differ from the first input musical piece;
perturbing the first set of musical characteristics to create a perturbed input vector as an input to a first set of nodes in a first visible layer of an unsupervised neural net, the perturbing for creating an output piece which differs from the first musical piece in the indicated manner, the unsupervised neural net comprised of a plurality of computing layers, each computing layer composed of a respective set of nodes;
using the perturbed input vector for training the unsupervised neural net in an unsupervised manner while calculating a higher level hidden layer in the unsupervised neural net; and
using the higher level hidden layer to create an output vector to create an output musical piece, wherein the output musical piece is a different musical piece which differs from the first input musical piece in the indicated manner.

2. The method as recited in claim 1 , wherein a subset of nodes in the first set of nodes represent musical characteristics at particular times in the first musical piece and the perturbing is performed by inserting values into a second set of nodes which are perturbation nodes in the first visible layer according to a rule selected according to the expressed user intent.

3. The method as recited in claim 2 , wherein in response to an expressed user intent, pitches having an interval from a note in the input piece are inserted into the set of perturbation nodes in the first visible layer.

4. The method as recited in claim 2 , further comprising:
receiving a user input indicating a degree of similarity for the output musical piece to the first musical input piece; and
wherein at least one of inserting values into the set of perturbation nodes or determining a length of training is performed according to the degree of similarity.

5. The method as recited in claim 2 , wherein the rule directs a selection of a set of pitches from a key signature associated with the user intent.

6. The method as recited in claim 2 , wherein the perturbing includes inserting random values into the set of perturbation nodes in the first visible layer.

7. The method as recited in claim 1 , further comprising:
receiving a second set of musical characteristics from a second input musical piece;
receiving user input that the output musical piece should more closely resemble the first input musical piece than the second input musical piece;
providing the second set of musical characteristics together with the first set of musical characteristics as the input vector; and
wherein the perturbing includes changing the perturbed vector so that the first input musical piece has a greater effect on the output musical piece than the second input musical piece by including the first set of musical characteristics in more nodes of the first set of nodes in the first visible layer of the unsupervised neural net than the second set of musical characteristics.

8. An apparatus, comprising:
a processor;
computer memory holding computer program instructions executed by the processor for generating a musical composition, the computer program instructions comprising:
program code operative to receive a first set of musical characteristics from a first input musical piece as an input vector and a user intent indicating a manner at least one of mood, length and genre of an output piece is to differ from the first input musical piece;
program code operative to perturb the first set of musical characteristics to create a perturbed input vector as an input to a first set of nodes in a first visible layer of an unsupervised neural net, the perturbing for creating an output piece which differs from the first musical piece in the indicated manner, the unsupervised neural net comprised of a plurality of computing layers, each computing layer composed of a respective set of nodes;
program code operative to use the perturbed input vector to train the unsupervised neural net in an unsupervised manner while calculating a higher level hidden layer in the unsupervised neural net; and
program code operative to use the higher level hidden layer to create an output vector to create an output musical piece;
wherein the output musical piece is a different musical piece which differs from the first input musical piece in the indicated manner.

9. The apparatus as recited in claim 8 , wherein the computer program instructions further comprise:
program code operative to perform the perturbing by inserting values into a set of perturbation nodes in the first visible layer according to a rule selected according to the expressed user intent, wherein other nodes in the first visible layer are used to represent musical characteristics at particular times in the first musical piece.

10. The apparatus as recited in claim 9 , wherein the computer program instructions further comprise program code operative to insert pitches having an interval from a note in the input piece are inserted into the set of perturbation nodes in the first visible layer in response to an expressed user intent.

11. The apparatus as recited in claim 9 , wherein the computer program instructions further comprise:
program code operative to receive a user input indicating a degree of similarity for the output musical piece to the first musical input piece; and
program code operative to insert values into the set of perturbation nodes according to the degree of similarity.

12. The apparatus as recited in claim 9 , wherein the computer program instructions further comprise program code operative to direct a selection of rule to insert a set of pitches from a key signature associated with the user intent into the first visible layer.

13. The apparatus as recited in claim 8 , wherein the computer program instructions further comprise:
program code operative to receive a second set of musical characteristics from a second input musical piece;
program code operative to receive user input that the output musical piece should more closely resemble the first input musical piece than the second input musical piece;
program code operative to provide the second set of musical characteristics together with the first set of musical characteristics as the input vector; and
program code operative to change the perturbed vector so that the first input musical piece has a greater effect on the output musical piece than the second input musical piece by including the first set of musical characteristics in more nodes of the first set of nodes in the first visible layer of the unsupervised neural net than the second set of musical characteristics.

14. A computer program product in a non-transitory computer readable medium for use in a data processing system, the computer program product holding computer program instructions which, when executed by the data processing system, for generating a musical composition, the computer program instructions comprising:
program code operative to receive a first set of musical characteristics from a first input musical piece as an input vector and a user intent indicating a manner at least one of mood, length and genre of an output piece is to differ from the first input musical piece;
program code operative to perturb the first set of musical characteristics to create a perturbed input vector as an inpu

Assignees

• IBM

Inventors

• Hwang Inseok
• Kuang Jente B
• Mukundan Janani

Classifications

• G10H2210/115

  using a random process to generate a musical note, phrase, sequence or structure · CPC title

• G10H2210/125

  Medley, i.e. linking parts of different musical pieces in one single piece, e.g. sound collage, DJ mix · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• G10H2250/311

  Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation · CPC title

• G10H2210/565

  Manual designation or selection of a tonality · CPC title