Machine learning model full life cycle management framework

Patent metadata

Field	Value
Publication number	US-11030086-B2
Application number	US-201916277057-A
Country	US
Kind code	B2
Filing date	Feb 15, 2019
Priority date	Feb 15, 2019
Publication date	Jun 8, 2021
Grant date	Jun 8, 2021

Abstract

A method and apparatus are provided that includes, in a same framework, storing an artificial intelligence (AI) model, loading the AI model into a serving platform, loading and testing a test unit against the AI model loaded into the serving platform, and collecting reports from results of storing the AI model, loading the AI model into the serving platform and testing the test unit.

First claim

What is claimed is:

1. An apparatus comprising
at least one memory configured to store computer program code;
at least one hardware processor configured to access said computer program code and operate as instructed by said computer program code, said computer program code including:
model storage code configured to cause the at least one hardware processor to store an artificial intelligence (AI) model;
model serving code configured to cause the at least one hardware processor to load the AI model into a serving platform from a plurality of serving platforms;
model testing code configured to cause the at least one hardware processor to load and test a test unit against the AI model loaded into the serving platform; and
monitoring and reporting code configured to cause the at least one hardware processor to collect reports from results of storing the AI model, loading the AI model into the serving platform and testing the test unit against the AI model loaded into the server platform.

2. The apparatus according to claim 1 , wherein the model storage code is further configured to cause the at least one hardware processor to remove the AI model from a storage.

3. The apparatus according to claim 1 , wherein the model serving code is further configured to cause the at least one hardware processor to periodically fetch a plurality of AI models.

4. The apparatus according to claim 3 , wherein the model serving code is further configured to cause the at least one hardware processor to periodically load the plurality of AI models into respective ones of a plurality of serving platforms.

5. The apparatus according to claim 1 , wherein the monitoring code is further configured to cause the at least one hardware processor to store the reports in respective ones of a plurality of databases.

6. The apparatus according to claim 1 , wherein the model storage code is configured to cause the at least one hardware processor to continuously check whether to store ones of a plurality of models.

7. The apparatus according to claim 1 , wherein the model serving code is configured to cause the at least one hardware processor to continuously check whether to load ones of a plurality of models into the serving platform.

8. The apparatus according to claim 1 , wherein the model serving code is configured to cause the at least one hardware processor to continuously check whether to load and test the test unit against ones of a plurality of models loaded into the serving platform.

9. The apparatus according to claim 1 , wherein the model monitoring and reporting code is configured to cause the at least one hardware processor to continuously check whether to provide ones of a plurality of reports.

10. The apparatus according to claim 1 , wherein the model monitoring and reporting code is configured to cause the at least one hardware processor to check whether to upgrade or to remove the AI model based on a result of testing the test unit against the AI model loaded into the serving platform.

11. A method comprising implementing in a framework:
storing an artificial intelligence (AI) model;
loading the AI model into a serving platform from a plurality of serving platforms;
loading and testing a test unit against the AI model loaded into the serving platform; and
collecting reports from results of storing the AI model, loading the AI model into the serving platform and testing the test unit against the AI model loaded into the server platform.

12. The method according to claim 11 , further comprising:
removing the AI model from a storage.

13. The method according to claim 11 , further comprising:
periodically fetching a plurality of AI models.

14. The method according to claim 13 , further comprising:
periodically loading the plurality of AI models into respective ones of a plurality of serving platforms.

15. The method according to claim 11 , further comprising:
storing the reports in respective ones of a plurality of databases.

16. The method according to claim 11 , further comprising:
continuously checking whether to store ones of a plurality of models.

17. The method according to claim 11 , further comprising:
continuously checking whether to load ones of a plurality of models into the serving platform.

18. The method according to claim 11 , further comprising:
continuously checking whether to load and test the test unit against ones of a plurality of models loaded into the serving platform.

19. The method according to claim 11 , further comprising:
continuously checking whether to provide ones of a plurality of reports.

20. A non-transitory computer readable medium storing a program causing a computer to execute a process, the process comprising:
storing an artificial intelligence (AI) model;
loading the AI model into a serving platform from a plurality of serving platforms;
loading and testing a test unit against the AI model loaded into the serving platform; and
collecting reports from results of storing the AI model, loading the AI model into the serving platform and testing the test unit against the AI model loaded into the server platform.

Assignees

• Tencent America LLC

Inventors

• Tu min
• Zhang Shangqing
• Yang Tao
• Li Yaliang
• Du Nan
• Xie Yusheng
• Fan Wei

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06F11/3698 Primary

  Environments for analysis, debugging or testing of software · CPC title