Customized predictive analytical model training

Patent metadata

Field	Value
Publication number	US-9342798-B2
Application number	US-201414295563-A
Country	US
Kind code	B2
Filing date	Jun 4, 2014
Priority date	Jun 27, 2011
Publication date	May 17, 2016
Grant date	May 17, 2016

Abstract

Methods, systems, and apparatus, including computer programs encoded on one or more computer storage devices, for training predictive models. Multiple training data records are received that each include an input data portion and an output data portion. A training data type is determined that corresponds to the training data. For example, a training data type can be determined by inputting the output data portions into one or more trained predictive classifiers. In other example, the training data type can be determined by comparison of the output data portions to data formats. Based on the determined training data type, a set of training functions are identified that are compatible with the training data of the determined training data type. The training data and the identified set of training functions are used to train multiple predictive models.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving a plurality of training data records, wherein each training data record includes an input data portion and an output data portion and data specifying a training data type that corresponds to the training records;
based on the training data type, identifying a set of one or more training functions that are included in a repository of training functions and that are compatible with the training data type, wherein identifying the set of training functions comprises:
inputting the training data type as input into one or more trained predictive models, each trained predictive model being trained to determine whether a category of training functions is compatible with the training data type;
receiving one or more predictive outputs from the one or more trained predictive models; and
identifying the set of training functions based on the one or more predictive outputs; and

using the training data records and the identified set of training functions obtained from the repository of training functions to train one or more predictive models.

2. The method of claim 1 , wherein identifying the set of training functions comprises:
identifying one or more constraints associated with the determined training data type; and
identifying one or more training functions that each satisfy at least one of the one or more constraints; wherein the identified one or more training functions comprise the set of training functions.

3. The method of claim 1 , further comprising:
generating a score for each of the one or more trained predictive models, where each score represents an estimation of the effectiveness of the respective trained predictive model; and
selecting a first trained predictive model from among the one or more trained predictive models based on the generated scores.

4. The method of claim 3 , wherein the one or more training data records are received over a network from a client computing system, and wherein the method further comprises:
providing access to the first trained predictive model to the client computing system over the network.

5. The method of claim 1 , further comprising:
inputting each output data portion as input to a trained predictive classifier;
receiving a plurality of predictive outputs from the trained predictive classifier, each predictive output comprising a label associated with a corresponding output data portion that was input to the trained predictive classifier; and
determining the training data type based on the plurality of labels provided by the trained predictive classifier.

6. The method of claim 5 , wherein:
the trained predictive classifier is trained to classify input as either numerical or not numerical;
training data that has all output data portions labeled as numerical is determined to be regression type training data; and
training data that has at least one output data portion labeled as non-numerical is determined to be classification type training data.

7. The method of claim 5 , wherein:
the trained predictive classifier is trained to classify input as either text or not text;
training data that has at least one output data portion labeled as text is determined to be classification type training data; and
training data that has all output data portions labeled as not text is determined to be regression type training data.

8. The method of claim 5 , further comprising:
inputting each output data portion as input to one or more additional trained predictive classifiers, wherein the one or more additional trained predictive classifiers are selected based on the determined training data type;
receiving a plurality of predictive outputs from the one or more additional trained predictive classifiers; and
based on the plurality of predictive outputs, determining a sub-type of the training data;
wherein identifying the set of training functions is further based on the determined sub-type of the training data.

9. A system comprising:
one or more data processing apparatus; and
a data store storing instructions that, when executed by the one or more data processing apparatus, cause the one or more data processing apparatus to perform operations comprising:
receiving a plurality of training data records, wherein each training data record includes an input data portion and an output data portion and data specifying a training data type that corresponds to the training records;
based on the training data type, identifying a set of one or more training functions that are included in a repository of training functions and that are compatible with the training data type, wherein identifying the set of training functions comprises:
inputting the training data type as input into one or more trained predictive models, each trained predictive model being trained to determine whether a category of training functions is compatible with the training data type;
receiving one or more predictive outputs from the one or more trained predictive models; and
identifying the set of training functions based on the one or more predictive outputs; and

using the training data records and the identified set of training functions obtained from the repository of training functions to train one or more predictive models.

10. The system of claim 9 , wherein identifying the set of training functions comprises:
identifying one or more constraints associated with the determined training data type; and
identifying one or more training functions that each satisfy at least one of the one or more constraints; wherein the identified one or more training functions comprise the set of training functions.

11. The system of claim 9 , wherein the operations further comprise:
generating a score for each of the one or more trained predictive models, where each score represents an estimation of the effectiveness of the respective trained predictive model; and
selecting a first trained predictive model from among the one or more trained predictive models based on the generated scores.

12. The system of claim 11 , wherein the one or more training data records are received over a network from a client computing system, and wherein the operations further comprise:
providing access to the first trained predictive model to the client computing system over the network.

13. The system of claim 9 , wherein the operations further comprise:
inputting each output data portion as input to a trained predictive classifier;
receiving a plurality of predictive outputs from the trained predictive classifier, each predictive output comprising a label associated with a corresponding output data portion that was input to the trained predictive classifier; and
determining the training data type based on the plurality of labels provided by the trained predictive classifier.

14. The system of claim 13 , wherein:
the trained predictive classifier is trained to classify input as either numerical or not numerical;
training data that has all output data portions labeled as numerical is determined to be regression type training data; and
training data that has at least one output data portion labeled as non-numerical is determined to be classification type training data.

15. The system of claim 13 , wherein:
the trained predictive classifier is trained to classify input as either text or not text;
training data that has at least one output data portion labeled as text is determined to be classification type training data; and
training data that has all output data portions labeled as not text is determined to be regress

Assignees

• Google Inc

Inventors

• Breckenridge Jordan M
• Green Travis H K
• Kaplow Robert
• Lin Wei-Hao
• Mann Gideon S

Classifications

• G06F18/285

  Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06K9/6262

  Physics · mapped topic

• G06F15/18

  Physics · mapped topic