Execution of a genetic algorithm having variable epoch size with selective execution of a training algorithm

Patent metadata

Field	Value
Publication number	US-2019073591-A1
Application number	US-201715697158-A
Country	US
Kind code	A1
Filing date	Sep 6, 2017
Priority date	Sep 6, 2017
Publication date	Mar 7, 2019
Grant date	—

Abstract

A method includes generating, by a processor of a computing device, a first plurality of models (including a first number of models) based on a genetic algorithm and corresponding to a first epoch of the genetic algorithm. The method includes determining whether to modify an epoch size for the genetic algorithm during a second epoch of the genetic algorithm based on a convergence metric associated with at least one epoch that is prior to the second epoch. The second epoch is subsequent to the first epoch. The method further includes, based on determining to modify the epoch size, generating a second plurality of models (including a second number of models that is different than the first number) based on the genetic algorithm and corresponding to the second epoch. Each model of the first plurality of models and the second plurality of models includes data representative of neural networks.

First claim

What is claimed is:

1 . A computer system comprising:
a memory configured to store an input data set; a processor configured to execute a recursive search, wherein executing the recursive search comprises:
generating a first plurality of data structures during a first iteration of the recursive search, wherein each of the first plurality of data structures includes data representative of a neural network, and wherein the first plurality of data structures includes a first number of data structures;
generating a second plurality of data structures based on at least one of the first plurality of data structures during a second iteration of the recursive search, wherein each of the second plurality of data structures includes data representative of a neural network,
wherein the second plurality of data structures includes a second number of data structures, wherein the second number is different than the first number, and wherein the second iteration is subsequent to the first iteration; and
providing a trainable data structure to an optimization trainer, the trainable data structure determined based on modifying one or more data structures of the second plurality of data structures, the optimization trainer configured to:
train the trainable data structure based on a portion of the input data set to generate a trained data structure; and
provide the trained data structure as input to a third iteration of the recursive search that is subsequent to the second iteration.

2 . The computer system of claim 1 , wherein executing the recursive search further comprises, during the second iteration of the recursive search:
selecting a subset of data structures from the second plurality of data structures based on fitness values associated with the subset of data structures, the fitness values determined based on at least a subset of the input data set; and performing at least one of a crossover operation or a mutation operation with respect to at least one data structure of the subset to generate the trainable data structure.

3 . The computer system of claim 1 , wherein the optimization trainer is executed on a different device, graphics processing unit (GPU), processor, core, thread, or any combination thereof, than the recursive search.

4 . A method comprising:
generating, by a processor of a computing device, a first plurality of models based on a genetic algorithm and corresponding to a first epoch of the genetic algorithm, wherein each of the first plurality of models includes data representative of a neural network, and wherein the first plurality of models includes a first number of models; determining whether to modify an epoch size during a second epoch of the genetic algorithm based on a convergence metric associated with at least one epoch that is prior to the second epoch, wherein the second epoch is subsequent to the first epoch; and based on a determination to modify the epoch size, generating a second plurality of models based on the genetic algorithm and corresponding to the second epoch, wherein each of the second plurality of models includes data representative of a neural network, wherein the second plurality of models includes a second number of models, and wherein the second number is different than the first number.

5 . The method of claim 4 , wherein the second number is less than the first number.

6 . The method of claim 4 , wherein the second number is greater than the first number.

7 . The method of claim 4 , wherein the convergence metric includes a fitness value corresponding to the at least one epoch or to at least one model of the at least one epoch, an improvement metric corresponding to the at least one epoch or to at least one model of the at least one epoch, a stagnation metric corresponding to the at least one epoch, or any combination thereof.

8 . The method of claim 4 , wherein the convergence metric includes an epoch number associated with the at least one epoch.

9 . The method of claim 4 , wherein the data representative of the neural network includes node data corresponding to a plurality of nodes of the neural network, connection data corresponding to one or more connections of the neural network, or any combination thereof.

10 . The method of claim 4 , further comprising:
providing a trainable model to an optimization trainer, the trainable model determined based on modifying one or more models of the second plurality of models; and adding a trained model received from the optimization trainer as input to a third epoch of the genetic algorithm that is subsequent to the second epoch.

11 . The method of claim 10 , further comprising:
determining a fitness value associated with the trained model; and providing a second trainable model to the optimization trainer based on the fitness value satisfying a threshold, the second trainable model determined based on modifying one or more models of a third plurality of models generated based on the genetic algorithm and corresponding to the third epoch.

12 . The method of claim 11 , wherein the fitness value associated with the trained model is determined based on a fitness function that is evaluated based on an input data set associated with the genetic algorithm.

13 . The method of claim 11 , further comprising providing a third trainable model to the optimization trainer based on the fitness value satisfying a second threshold, the third trainable model determined based on modifying one or more models of the third plurality of models.

14 . The method of claim 10 , further comprising:
determining a fitness value associated with the trained model; and refraining from providing any trainable models to the optimization trainer for at least one epoch based on the fitness value failing to satisfy a threshold.

15 . The method of claim 14 , further comprising providing a second trainable model to the optimization trainer, the second trainable model based on identifying one or more models of a third plurality of models generated based on the genetic algorithm and corresponding to a fourth epoch of the genetic algorithm, wherein the fourth epoch is subsequent to the third epoch, and wherein the third epoch and the fourth epoch are separated by at least one epoch of the genetic algorithm.

16 . The method of claim 10 , wherein the optimization trainer is configured to update connection weights of the trainable model but not a topology or activation functions of the trainable model.

17 . The method of claim 10 , wherein modifying the one or more models of the second plurality of models includes performing at least one of a crossover operation or a mutation operation with respect to the one or more models of the second plurality of models.

18 . A computer-readable storage device storing instructions that, when executed, cause a computer to perform operations comprising:
determining a trainable model to provide to a trainer, the trainable model determined based on modification of one or more models of a first plurality of models that is generated based on a genetic algorithm and corresponding to a first epoch of the genetic algorithm, wherein each of the first plurality of models includes data representative of a neural network; providing the trainable model to the trainer; adding a trained model received from the trainer as input to a second epoch of the genetic algorithm that is subsequent to the first epoch; determining whether to enable the trainer, disable the trainer, or activate at least one additional trainer for a third epoch of the g

Assignees

• Sparkcognition Inc

Inventors

• Andoni Sari
• Moore Keith D
• Bonab Elmira M
• Choi Junhwan

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/048

  Activation functions · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N20/10

  using kernel methods, e.g. support vector machines [SVM] · CPC title

• G06N3/006

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title