Automated model building search space reduction

Patent metadata

Field	Value
Publication number	US-10657447-B1
Application number	US-201816205088-A
Country	US
Kind code	B1
Filing date	Nov 29, 2018
Priority date	Nov 29, 2018
Publication date	May 19, 2020
Grant date	May 19, 2020

Abstract

A method includes receiving, by a processor, an input data set. The input data set includes a plurality of features. The method includes determining, by the processor, one or more characteristics of the input data set. The method includes, based on the one or more characteristics, adjusting, by the processor, one or more architectural parameters of an automated model generation process. The automated model generation process is configured to generate a plurality of models using a weighted randomization process. The one or more architectural parameters weight the weighted randomization process to adjust a probability of generation of models having particular architectural features. The method further includes executing, by the processor, the automated model generation process to output a mode, the model including data representative of a neural network.

First claim

What is claimed is:

1. A method of neural network generation, the method comprising:
receiving, by a processor, an input data set, the input data set including a plurality of features;
determining, by the processor, one or more characteristics of the input data set;
based on the one or more characteristics, adjusting, by the processor, one or more architectural parameters of an automated model generation process, wherein the automated model generation process is configured to generate a plurality of models using a weighted randomization process, wherein the one or more architectural parameters weight the weighted randomization process to adjust a probability of generation of models having particular architectural features, and wherein adjusting the one or more architectural parameters includes setting a first architectural parameter to a first value, the first architectural parameter associated with a probability that models of a first epoch of the weighted randomization process have a first model type; and
executing, by the processor, the automated model generation process to output a model, the model including data representative of a neural network.

2. The method of claim 1 , wherein the particular architectural features comprise an initial model type used by the weighted randomization process.

3. The method of claim 2 , wherein the initial model type comprises feedforward models, recurrent models, pooling-based two-dimensional convolutional models, daisy-chains of causal convolutional models, or a combination thereof.

4. The method of claim 1 , wherein the one or more architectural parameters include a mutation parameter, and wherein at least one model of the plurality of models generated using the weighted randomization process is modified based on the mutation parameter.

5. The method of claim 1 , further comprising, based on the one or more characteristics, adjusting, by the processor, one or more training hyperparameters of the automated model generation process, wherein the one or more training hyperparameters control one or more aspects of training of the model.

6. The method of claim 1 , wherein adjusting the one or more architectural parameters further includes:
setting a second architectural parameter to a second value based on the one or more characteristics, the second architectural parameter associated with a probability that models of the first epoch of the weighted randomization process have a second model type.

7. The method of claim 1 , wherein the one or more characteristics indicate a type of problem associated with the input data set, a data type associated with the input data set, or a combination thereof.

8. The method of claim 1 , wherein adjusting the one or more architectural parameters based on the one or more characteristics comprises comparing the one or more characteristics to a set of rules that maps data set characteristics to architectural parameters, wherein the set of rules maps the data set to characteristics of grammars, and wherein the grammars indicate corresponding architectural parameters.

9. The method of claim 8 , further comprising updating the set of rules based on characteristics of the model.

10. The method of claim 9 , wherein the set of rules are updated responsive to a score of the model satisfying a threshold.

11. The method of claim 1 , wherein adjusting the one or more architectural parameters based on the one or more characteristics comprises providing data indicative of the one or more characteristics to a particular neural network configured to identify one or more architectural parameters for adjustment based on the data indicative of the one or more characteristics.

12. The method of claim 11 , further comprising retraining the particular neural network based on training data, the training data indicating characteristics of the model.

13. The method of claim 1 , wherein executing the automated model generation process comprises:
based on a fitness function, selecting, by the processor, a subset of models from the plurality of models, the plurality of models based on a genetic algorithm and corresponding to a first epoch of the genetic algorithm;
performing, by the processor, at least one genetic operation of the genetic algorithm with respect to at least one model of the subset to generate a trainable model;
sending the trainable model to an optimization trainer; and
adding a trained model received from the optimization trainer as input to a second epoch of the genetic algorithm that is subsequent to the first epoch.

14. The method of claim 13 , wherein the fitness function is evaluated based on the input data set, and wherein the optimization trainer is configured to use a portion of the input data set to train the trainable model.

15. A computing device comprising:
a processor; and
a memory storing instructions executable by the processor to perform operations comprising:
receiving an input data set, the input data set including a plurality of features;
determining one or more characteristics of the input data set;
based on the one or more characteristics, adjusting one or more architectural parameters of an automated model generation process, wherein the automated model generation process is configured to generate a plurality of models using a weighted randomization process, wherein the one or more architectural parameters weight the weighted randomization process to adjust a probability of generation of models having particular architectural features, and wherein adjusting the one or more architectural parameters includes setting a first architectural parameter to a first value, the first architectural parameter associated with a probability that models of a first epoch of the weighted randomization process have a first model type; and
executing the automated model generation process to output a model, the model including data representative of a neural network.

16. The computing device of claim 15 , wherein the particular architectural features comprise an initial model type used by the weighted randomization process, and wherein the initial model type comprises feedforward models, recurrent models, pooling-based two-dimensional convolutional models, daisy-chains of convolutional models, or a combination thereof.

17. The computing device of claim 15 , wherein the one or more characteristics indicate a type of problem associated with the input data set, a data type associated with the input data set, or a combination thereof.

18. A computer-readable storage device storing instructions that, when executed by a processor, cause the processor to perform operations comprising:
receiving an input data set, the input data set including a plurality of features;
determining one or more characteristics of the input data set;
based on the one or more characteristics, adjusting one or more architectural parameters of an automated model generation process, wherein the automated model generation process is configured to generate a plurality of models using a weighted randomization process, wherein the one or more architectural parameters weight the weighted randomization process to adjust a probability of generation of models having particular architectural, and wherein adjusting the one or more architectural parameters includes setting a first architectural parameter to a first value, the first architectural parameter associated with a probability that models of a first epoch of the weighted randomization process have a first model type; and
executing the automated model generation process to o

Assignees

• Sparkcognition Inc

Inventors

• Mcdonnell Tyler S
• Andoni Sari
• Choi Junhwan
• Goode Jimmie
• Lan Yiyun
• Moore Keith D
• Sellers Gavin

Classifications

• G06N3/086 Primary

  using evolutionary algorithms, e.g. genetic algorithms or genetic programming · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/0445

  Physics · mapped topic

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/045

  Combinations of networks · CPC title