Adjusting automated neural network generation based on evaluation of candidate neural networks

Patent metadata

Field	Value
Publication number	US-2019122119-A1
Application number	US-201715793890-A
Country	US
Kind code	A1
Filing date	Oct 25, 2017
Priority date	Oct 25, 2017
Publication date	Apr 25, 2019
Grant date	—

Abstract

A method includes determining, by a processor of a computing device, an expected performance or reliability of a first neural network of a first plurality of neural networks. The expected performance or reliability is determined based on a vector representing at least a portion of the first neural network, where the first neural network is generated based on an automated generative technique (e.g., a genetic algorithm) and where the first plurality of neural networks corresponds to a first epoch of the automated generative technique. The method also includes responsive to the expected performance or reliability of the first neural network failing to satisfy a threshold, adjusting a parameter of the automated generative technique. The method further includes, during a second epoch of the automated generative technique, generating a second plurality of neural networks based at least in part on the adjusted parameter.

First claim

1 . A computer system for generating a neural network, the computer system comprising:
a memory configured to store a plurality of data structures, each of the plurality of data structures including data representative of a respective neural network of a first plurality of neural networks; and a processor configured to execute a recursive search based on a set of parameters,
wherein executing the recursive search comprises, during a first iteration:
generating a vector representing at least a portion of a first neural network of the first plurality of neural networks;
inputting the vector to a trained classifier to generate output indicating an expected performance or reliability of the first neural network; and
responsive to the expected performance or reliability of the first neural network failing to satisfy a threshold, adjusting at least one parameter of the set;
wherein executing the recursive search comprises, during a second iteration, generating a second plurality of neural networks based on the at least one adjusted parameter; wherein the recursive search is executed until a termination criterion is reached, wherein a third plurality of neural networks are generated during a final iteration of the recursive search at least in part based on the second plurality of neural networks; and wherein executing the recursive search comprises outputting one or more neural networks of the third plurality of neural networks.

2 . The computer system of claim 1 , wherein the expected performance or reliability of the first neural network is determined at a different device, graphics processing unit (GPU), processor, core, thread, or any combination thereof, than execution of the recursive search.

3 . A method for generating a neural network, the method comprising:
inputting, by a processor of a computing device, a vector to a trained classifier to generate output indicating an expected performance or reliability of a first neural network of a first plurality of neural networks, the vector representing at least a portion of the first neural network, wherein the first neural network is generated based on an automated generative technique and wherein the first plurality of neural networks corresponds to a first epoch of the automated generative technique; responsive to the expected performance or reliability of the first neural network failing to satisfy a threshold, adjusting a parameter of the automated generative technique to increase a likelihood of at least one preferred neural network characteristic being included in a second plurality of neural networks generated during a second epoch of the automated generative technique, decrease a likelihood of at least one disfavored neural network characteristic being included in the second plurality of neural networks, or both; executing the automated generative technique until a termination criterion is reached, wherein a third plurality of neural networks are generated during a final epoch of the automated generative technique at least in part based on the second plurality of neural networks; and outputting one or more neural networks of the third plurality of neural networks as an output of the automated generative technique.

4 . The method of claim 3 , wherein the automated generative technique comprises a genetic algorithm, and further comprising, during a second epoch of the genetic algorithm, generating the second plurality of neural networks based at least in part on the adjusted parameter.

5 . The method of claim 3 , wherein the vector is determined based on a data structure representing the first neural network, the data structure including a plurality of fields including values representing topology of the first neural network.

6 . The method of claim 5 , wherein the plurality of fields is descriptive of a first node of the first neural network, a second node of the first neural network, and a link between the first node and second node.

7 . The method of claim 6 , wherein the plurality of fields indicates at least one of an activation function, an aggregation function, or a bias function of the first node.

8 . The method of claim 6 , wherein the plurality of fields indicates a link weight of the link between the first node and the second node.

9 . The method of claim 3 , wherein the vector is a normalized vector that includes a set of values arranged based on a sequence of input nodes of a trained classifier.

10 . The method of claim 3 , wherein the vector includes one or more values indicating a topology of the first neural network.

11 . The method of claim 3 , further comprising determining whether the first neural network satisfies a similarity threshold with respect to a second neural network.

12 . The method of claim 11 , wherein determining whether the first neural network satisfies the similarity threshold with respect to the second neural network comprises determining a binned hamming distance between the vector and a second vector representing at least a portion of the second neural network.

13 . canceled

14 . The method of claim 3 , wherein the parameter comprises a mutation parameter of a genetic algorithm, and wherein the mutation parameter determines at least one of a mutation likelihood, an extent of mutation, or a type of mutation of each candidate neural network of an epoch of the genetic algorithm.

15 . The method of claim 14 , wherein the mutation parameter is specific to the first neural network, neural networks generated based on crossover operations that involve the first neural network, or any combination thereof.

16 . The method of claim 3 , wherein adjusting the parameter comprises removing the first neural network from a population of neural networks associated with a genetic algorithm.

17 . A computer-readable storage device storing instructions that, when executed, cause a computer to perform operations comprising:
determining an expected performance or reliability of a first neural network of a first plurality of neural networks, the expected performance or reliability determined based on a vector representing at least a portion of the first neural network, wherein the first neural network is generated based on a genetic algorithm and wherein the first plurality of neural networks corresponds to a first epoch of the genetic algorithm; responsive to the expected performance or reliability of the first neural network failing to satisfy a threshold, adjusting a parameter of the genetic algorithm; during a second epoch of the genetic algorithm, generating a second plurality of neural networks based at least in part on the adjusted parameter; performing the genetic algorithm until a termination criterion is reached, wherein a third plurality of neural networks are generated during a final epoch of the genetic algorithm based at least in part on the second plurality of neural networks; and outputting one or more neural networks of the third plurality of neural networks as an output of the genetic algorithm.

18 . The computer-readable storage device of claim 17 , wherein determining the expected performance or reliability of the first neural network comprises providing the vector as input to a trained classifier to generate a classification result associated with at least the portion of the first neural network, the classification result indicative of the expected performance or reliability.

19 . The computer-readable storage device of claim 17 , wherein determining the expected performance or reliability of the first neural network co

Assignees

• Sparkcognition Inc

Inventors

• Husain Syed Mohammad Amir

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/048

  Activation functions · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/047

  Probabilistic or stochastic networks · CPC title