Execution of a genetic algorithm with variable evolutionary weights of topological parameters for neural network generation and training

Patent metadata

Field	Value
Publication number	US-11106978-B2
Application number	US-201715699353-A
Country	US
Kind code	B2
Filing date	Sep 8, 2017
Priority date	Sep 8, 2017
Publication date	Aug 31, 2021
Grant date	Aug 31, 2021

Abstract

A method includes generating, by a processor of a computing device, an output set of models corresponding to a first epoch of a genetic algorithm and based on an input set of models of the first epoch. The input set and the output set includes data representative of a neural network. The method includes determining a particular model of the output set based on a fitness function. A first topological parameter of a first model of the input set is modified to generate the particular model of the output set. The method includes modifying a probability that the first topological parameter is to be changed by a genetic operation during a second epoch of the genetic algorithm that is subsequent to the first epoch. The method includes generating a second output set of models corresponding to the second epoch and based on the output set and the modified probability.

First claim

What is claimed is:

1. A computer system comprising:
a memory configured to store an input data set and a first plurality of data structures, each of the first plurality of data structures including data representative of a neural network;
a processor configured to execute a recursive search, wherein executing the recursive search comprises:
during a first iteration of the recursive search, determining a fitness value for each of the first plurality of data structures based on at least a subset of the input data set;
during a second iteration of the recursive search:
generating a second plurality of data structures based on the first plurality of data structures; and
determining a fitness value for each of the second plurality of data structures based on at least the subset of the input data set;

comparing the fitness values of each of the second plurality of data structures to the fitness values of each of the first plurality of data structures to determine a particular data structure of the second plurality of data structures;
determining a difference in at least one topological parameter between the particular data structure of the second plurality of data structures and a parent data structure of the first plurality of data structures;
modifying a probability that the at least one topological parameter is to be changed during a third iteration of the recursive search to a modified probability; and
during the third iteration, generating a third plurality of data structures based at least in part on the modified probability.

2. The computer system of claim 1 , wherein the at least one topological parameter includes a number of nodes, a number of connections, a number of input nodes, a number of hidden layers, or a combination thereof.

3. A method comprising:
generating, by a processor of a computing device, an output set of models corresponding to a first epoch of a genetic algorithm, the output set of models based on an input set of models of the first epoch, wherein each of the input set of models and the output set of models includes data representative of a neural network;
determining a particular model of the output set of models based on a fitness function, wherein a first topological parameter of a first model of the input set of models is modified to generate the particular model of the output set of models;
determining a second particular model of the output set of models based on the fitness function, wherein a second topological parameter of a second model of the input set of models is modified to generate the second particular model of the output set of models;
modifying a probability that the first topological parameter is to be changed by a genetic operation during a second epoch of the genetic algorithm that is subsequent to the first epoch to a first probability;
modifying a probability that the second topological parameter is to be changed by the genetic algorithm during the second epoch to a second probability; and
generating a second output set of models corresponding to the second epoch, the second output set of models based on the output set of models, the first probability, and the second probability.

4. The method of claim 3 , wherein the output set of models are provided as a second input set of models to the second epoch, and wherein the genetic operation includes at least one mutation operation.

5. The method of claim 3 , further comprising determining a particular modification of the first topological parameter associated with generation of the particular model of the output set of models, wherein modifying the probability that the first topological parameter is to be changed comprises modifying a probability that the genetic operation performs the particular modification to the first topological parameter during the second epoch.

6. The method of claim 3 , further comprising:
generating a first fitness value associated with one or more of the input set of models based on the fitness function;
generating a second fitness value associated with the particular model of the output set of models based on the fitness function; and
comparing a difference between the second fitness value and the first fitness value to a threshold.

7. The method of claim 6 , wherein modifying the probability that the first topological parameter is to be changed comprises increasing the probability based on the difference satisfying the threshold and having a positive magnitude.

8. The method of claim 6 , wherein modifying the probability that the first topological parameter is to be changed comprises decreasing the probability based on the difference satisfying the threshold and having a negative magnitude.

9. The method of claim 6 , wherein the first fitness value comprises an average fitness value for a first species of models, the first species of models including the first model and the particular model.

10. The method of claim 6 , further comprising:
identifying a second particular model of the second output set of models, wherein the second particular model is generated based on performance of the genetic operation on the particular model;
determining a third fitness value associated with the second particular model based on the fitness function;
comparing a difference between the third fitness value and the second fitness value to the threshold; and
modifying a particular probability that the first topological parameter is to be changed by a second genetic operation during a third epoch of the genetic algorithm that is subsequent to the second epoch based on the difference between the third fitness value and the second fitness value failing to satisfy the threshold.

11. The method of claim 10 , wherein, after modifying the particular probability, a difference between the particular probability and an initial probability is less than a difference between the probability that the first topological parameter is to be changed and the initial probability.

12. The method of claim 3 , wherein the first epoch and the second epoch are consecutive epochs.

13. The method of claim 3 , further comprising:
determining a second topological parameter of the first model that is modified to generate the particular model of the output set of models; and
modifying a particular probability that the second topological parameter is to be changed by the genetic operation during the second epoch.

14. The method of claim 3 , wherein the first topological parameter includes a number of nodes, and wherein modifying the probability that the first topological parameter is to be changed comprises adjusting a probability that the genetic operation changes the number of nodes in at least one model of the second output set of models.

15. The method of claim 3 , wherein the first topological parameter includes a number of connections, and wherein modifying the probability that the first topological parameter is to be changed comprises adjusting a probability that the genetic operation changes the number of connections in at least one model of the second output set of models.

16. The method of claim 3 , further comprising determining whether to modify a node parameter associated with a particular node of a particular model of the second output set of models during a third epoch of the genetic algorithm based on a fitness value associated with the particular model of the second output set of models.

17. The method of claim 16 , wherein the node parameter includes an activation function, an aggregation function, a bias function, or a combination thereof.

18.

Assignees

• Sparkcognition Inc

Inventors

• Andoni Sari
• Moore Keith D
• Bonab Elmira M
• Choi Junhwan
• Korman Eric O

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/048

  Activation functions · CPC title

• G06F16/24566

  Recursive queries · CPC title

• G06N3/006

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title