Computer-based systems and methods configured to utilize automating deployment of predictive models for machine learning tasks

Patent metadata

Field	Value
Publication number	US-10853418-B1
Application number	US-201916511611-A
Country	US
Kind code	B1
Filing date	Jul 15, 2019
Priority date	Jul 12, 2019
Publication date	Dec 1, 2020
Grant date	Dec 1, 2020

Abstract

A method includes obtaining feature generation code from, which is configured to determine features relating to input data. The method further includes obtaining data grouping code, which is configured to generate training data by determining a plurality of data groupings for the features relating to the input data. The method further includes obtaining modeling code, which is derived at least in part by applying one or more machine learning algorithms to the training data. The method further includes applying a model wrapper code to the feature generation code, the data grouping code, and the modeling code to generate a model wrapper and deploying the model wrapper such that the model wrapper may receive a first application programming interface (API) call including an input data value, determine a score relating to the input data value, and send a second API call including the score in response to the first API call.

First claim

What is claimed is:

1. A method comprising:
building, by one or more processors, feature generation code and deploying the feature generation code to a repository;
receiving, by the one or more processors, input data;
determining, by the one or more processors using the feature generation code, features relating to the input data;
building, by the one or more processors, data grouping code and deploying the data grouping code to the repository,
generating, by the one or more processors using the data grouping code, training data by:
determining a plurality of data groupings for the features relating to the input data, wherein the plurality of data groupings comprises a wildcard data grouping, and assigning at least one unrecognized feature relating to the input data to the wildcard data grouping; and

building, by the one or more processors, modeling code and deploying the modeling code to the repository, wherein the modeling code is derived at least in part by applying one or more machine learning algorithms to the training data, wherein:
the feature generation code, the data grouping code, and the modeling code are configured to be pulled from the repository by a model wrapper code; and
the model wrapper code is configured to generate a model wrapper configured to, after deployment of the model wrapper:
receive a first application programming interface (API) call comprising an input data value,
determine a score relating to the input data value, and
send a second API call comprising the score in response to the first API call.

2. The method of claim 1 , further comprising generating, by the one or more processors, modeling code configuration files comprising information about the format and contents of the modeling code.

3. The method of claim 2 , further comprising deploying the modeling code configuration files to the repository along with the modeling code.

4. The method of claim 1 , further comprising:
sending, by the one or more processors after determining the features relating to the input data, an approval request to a supervisor electronic device; and
receiving, by the one or more processors from the supervisor electronic device, an approval of the features relating to the input data in response to the approval request.

5. The method of claim 4 , wherein the training data is generated only after the approval is received.

6. The method of claim 1 , wherein the model wrapper is configured to be deployed by:
sending, by the one or more processors, the model wrapper to a model wrapper repository; and
applying, by the one or more processors, an API code to the model wrapper to generate an API image configured to send and receive API calls.

7. The method of claim 1 , wherein the input data value and the input data comprise a same type of data.

8. The method of claim 1 , further comprising receiving, by the one or more processors, updated code comprising one or two of, but not all three of, updated feature generation code, updated data grouping code, or updated modeling code.

9. The method of claim 8 , further comprising building, by the one or more processors, the updated code and deploying the updated code to the repository.

10. A system comprising:
a memory;
at least one processor coupled to the memory, the processor configured to:
build feature generation code and deploy the feature generation code to a repository;
receive input data;
determine, using the feature generation code, features relating to input data;
build data grouping code and deploy the data grouping code to the repository,
generate, using the data grouping code, training data, wherein the data grouping code is configured to determine a plurality of data groupings for the features relating to the input data;
build modeling code and deploy the modeling code to the repository;
generate modeling code configuration files comprising information about the format and contents of the modeling code, wherein:
the modeling code is derived at least in part by applying one or more machine learning algorithms to the training data;
the feature generation code, the data grouping code, the modeling code, and the modeling code configuration files are configured to be pulled from the repository by a model wrapper code; and
the model wrapper code is configured to generate a model wrapper configured to, after deployment of the model wrapper:
receive a first application programming interface (API) call comprising an input data value,
determine a score relating to the input data value, and
send a second API call comprising the score in response to the first API call.

11. The system of claim 10 , wherein the modeling code configuration files are generated at least in part based on the training data.

12. The system of claim 10 , wherein the model wrapper code is configured to be applied to generate the model wrapper based at least in part on the modeling code configuration files.

13. The system of claim 10 , wherein the modeling code configuration files are generated based at least in part on the modeling code.

14. The system of claim 10 , wherein the modeling code is deployed along with the modeling code configuration files in the repository.

15. The system of claim 10 , wherein the one or more machine learning algorithms comprise a plurality of machine learning algorithms, and further wherein the processor is configured to:
receive a selection of at least one of the plurality of machine learning algorithms with which to generate the modeling code.

16. The system of claim 15 , wherein the processor is further configured to generate the modeling code based at least in part on the selected at least one of the plurality of machine learning algorithms.

17. A non-transitory computer readable medium having instructions stored thereon that, upon execution by a computing device, cause the computing device to perform operations comprising:
building feature generation code and deploying the feature generation code to a repository;
receiving input data;
determining, using the feature generation code, features relating to input data;
building data grouping code and deploying the data grouping code to the repository,
generating, using the data grouping code, training data by determining a plurality of data groupings for the features relating to the input data; and
building modeling code and deploying the modeling code to the repository, wherein the modeling code is derived at least in part by applying one or more machine learning algorithms to the training data, wherein:
the feature generation code, the data grouping code, and the modeling code are configured to be pulled from the repository by a model wrapper code; and
the model wrapper code is configured to generate a model wrapper configured to, after deployment of the model wrapper:
receive a first application programming interface (API) call comprising an input data value,
determine a score relating to the input data value, and
send a second API call comprising the score in response to the first API call.

18. The non-transitory computer readable medium of claim 17 , wherein the model wrapper is configured to be deployed by applying an API code to the model wrapper.

19. The non-transitory computer readable medium of claim 17 , wherein the plurality of data groupings comprises a wildcard data grouping, and wherein the computing device is further configured to assign at least one unrecognized feature relating to the input data to the wildcard data grouping.

20.

Assignees

• Capital One Services Llc

Inventors

• Wu Chen

Classifications

• G06F8/60 Primary

  Software deployment · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/2178

  based on feedback of a supervisor · CPC title

• G06F16/38

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title