Automated machine learning tool

Patent metadata

Field	Value
Publication number	US-2017193392-A1
Application number	US-201514986596-A
Country	US
Kind code	A1
Filing date	Dec 31, 2015
Priority date	Dec 31, 2015
Publication date	Jul 6, 2017
Grant date	—

Abstract

Techniques are provided for generating and deploying a computer model with relatively few inputs from a user. Techniques are also provided for creating a data mart that multiple computer models may leverage in order to decrease the time required to generate subsequent computer models.

First claim

What is claimed is:

1 . A method comprising:
prior to receiving input to generate a plurality of models, storing multiple sets of parameter values for a set of parameters upon which the plurality of models are to be based; after storing the multiple sets of parameter values, determining a set of features upon which the plurality of models will be based; for each set of parameter values of the multiple sets of parameter values, generating a different model of the plurality of models based on the set of features, training data, and said each set of parameter values; for each model of the plurality of models, determining a performance metric for said each model; based on the performance metric for each model of the plurality of models, selecting a particular model from among the plurality of models to deploy; wherein the method is performed by one or more computing devices.

2 . The method of claim 1 , wherein, for each model of the plurality of models, the set of parameter values that corresponds to said each model does not change while said each model is generated.

3 . The method of claim 1 , wherein determining the performance metric for said each model comprises:
applying said each model to validation data to generate a set of results; comparing the set of results to a set of known results.

4 . The method of claim 1 , wherein the performance metric of each model is an area under the curve (AUC) metric.

5 . The method of claim 1 , wherein the set of parameters comprise one or more of a learning rate, regularization, or number of iterations.

6 . The method of claim 1 , wherein generating the plurality of models comprises generating each model of the plurality of models concurrently with each other model of the plurality of models.

7 . The method of claim 1 , wherein no user input is received between generating the plurality of models and selecting the particular model.

8 . The method of claim 1 , further comprising:
creating, by a first service, a job that indicates generation of multiple models; sending the job from the first service to a cluster computing system; wherein generating the plurality of models comprises executing the job by the cluster computing system.

9 . The method of claim 1 , wherein:
a first user established the multiple sets of parameter values; the method comprising receiving input from a second user that is different the first user; generating the plurality of models comprises generating the plurality of models based on the input.

10 . A method comprising:
prior to receiving input to generate a computer model, storing operation data that identifies a plurality of transformation operations; after storing the operation data, identifying a plurality of features; in response to identifying the plurality of features, for each feature of the plurality of features:
applying, based on the operation data, the plurality of transformation operations to said each feature to generate a plurality of transformed features of said each feature;
storing the plurality of transformed features of said each feature in a particular set of features;
after the plurality of transformed features of said each feature are stored in the particular set of features, using the particular set of features and training data to generate a plurality of computer models; wherein the method is performed by one or more computing devices.

11 . The method of claim 10 , further comprising:
automatically selecting all the features in the particular set of features for generating the plurality of computer models, wherein no user input is received between identifying the plurality of features and using the particular set of features and the training data to generate the plurality of computer models.

12 . The method of claim 10 , further comprising:
automatically selecting all the features in the particular set of features for generating the plurality of computer models; prior to using the particular set of features and the training data to generate the plurality of computer models, causing, to be displayed, a user interface that allows a user to remove one or more features from the particular set of features.

13 . The method of claim 10 , further comprising:
after generating the plurality of computer models, causing, to be displayed, a user interface that allows a user to remove one or more features from the particular set of features; receiving, through the user interface, input that indicates one or more particular features to remove from the particular set of features; in response to receiving the input, removing the one or more particular features from the particular set of features; after removing the one or more particular features from the particular set of features, using the particular set of features and the training data to generate a second plurality of computer models.

14 . The method of claim 10 , wherein the plurality of transformation operations comprise one or more of sum, mean, median, maximum, minimum, binary, or logarithm.

15 . A method comprising:
automatically generating, based on first training data, a first model that is used to predict a particular event; deploying the first model; after generating the first model and without receiving input from any user, automatically generating, based on second training data that is different than the first training data, a second model that is different than the first model and that is used to predict the particular event; deploying the second model; wherein the method is performed by one or more computing devices.

16 . The method of claim 15 , wherein deploying the second model comprises deploying the second model while the first model is deployed.

17 . The method of claim 16 , further comprising, while the first model and the second model are deployed concurrently, assigning a first portion of entities to the first model and assigning a second portion of the entities to the second model.

18 . The method of claim 17 , further comprising:
while the first model and the second model are deployed concurrently, determining a first performance metric for the first model and a second performance metric for the second model; performing a comparison between the first performance metric and the second performance metric; based on the comparison, determining to increase a number of entities that will be assigned to the second model.

19 . The method of claim 17 , wherein the entities are multiple users of an online system, wherein each user of the multiple users interact with the online system using a computing device that communicatively connects to the online system.

20 . The method of claim 15 , wherein deploying the second model comprises automatically deploying the second model after generating the first model and without receiving input from any user.

21 . The method of claim 15 , wherein:
the first training data is based on first conversion data that indicates a first set of conversions that occurred over a first time period; the method further comprising identifying second conversion data that indicates a second set of conversions that occurred over a second period of time that is different than the first time period; the second training data is based on the second conversion data.

Assignees

• Linkedin Corp

Inventors

• Liu Yan
• Zhu Qiang
• Wang Guan
• Guo Songtao
• Liu Shaobo

Classifications

• G06F30/20

  Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30) · CPC title

• G06N99/005 Primary

  Physics · mapped topic

• G06F17/5009

  Physics · mapped topic

• G06N5/04

  Inference or reasoning models · CPC title

• G06N20/00 Primary

  Machine learning · CPC title