Management of artificial intelligence development platform

Patent metadata

Field	Value
Publication number	US-11455173-B2
Application number	US-202117207268-A
Country	US
Kind code	B2
Filing date	Mar 19, 2021
Priority date	Jun 30, 2020
Publication date	Sep 27, 2022
Grant date	Sep 27, 2022

Abstract

A method for management of an artificial intelligence development platform is provided. The artificial intelligence development platform is deployed with instances of a plurality of model services, and each of the model services is provided with one or more instances. The method includes: acquiring calling information of at least one model service; determining the activity of the at least one model service according to the calling information; and at least deleting all instances of the at least one model service in response to that the determined activity meets a first condition.

First claim

The invention claimed is:

1. A method for management of an artificial intelligence, AI, development platform, the AI development platform being deployed with instances of a plurality of AI model services, and each of the plurality of AI model services being provided with one or more instances, wherein the method comprises:
acquiring calling information of all instances of at least one AI model service of the plurality of AI model services;
determining calling activity of the at least one AI model service according to the calling information; and
deleting all instances of the at least one AI model service without deleting a routing configuration of the at least one AI model service in an access layer of the AI development platform in response to the determined calling activity meeting a calling frequency within the predetermined time window.

2. The method of claim 1 , further comprising:
in response to determining that the at least one AI model service of which the instances have been deleted is not called within a predetermined period, deleting the routing configuration of the at least one AI model service in the access layer of the AI development platform.

3. The method of claim 1 , wherein deleting all instances of the at least one AI model service comprises:
determining whether the at least one AI model service exists in a cluster of the AI development platform; and
in response to determining that the at least one AI model service exists in the cluster, switching a backend of the at least one AI model service to a traffic receiving module, wherein the traffic receiving module is used for receiving a model reasoning traffic for the AI model service of which the instances have been deleted.

4. The method of claim 3 , wherein deleting all instances of the at least one AI model service further comprises:
continuing to store original data information of the at least one AI model service into the cluster, wherein the original data information at least comprises a first number of the instances deployed on the AI development platform before the instances of the at least one AI model service is deleted.

5. The method of claim 4 , further comprising:
in response to the traffic receiving module receives the model reasoning traffic of the at least one AI model service of which the instances have been deleted, triggering a wakeup operation for the at least one AI model service, wherein the wakeup operation comprises:
recovering the first number of instances of the at least one AI model service on the AI development platform, and polling states of the recovered instances until the starting of the first number of instances is completed; and
switching the backend of the at least one AI model service back to the first number of recovered instances.

6. The method of claim 5 , wherein in the case that the routing configuration of the at least one AI model service in the access layer of the AI development platform has been deleted, the wakeup operation further comprises:
reconfiguring a routing rule corresponding to the at least one AI model service.

7. The method of claim 5 , wherein the traffic receiving module has a plurality of instances, wherein
when a first instance in the plurality of instances of the traffic receiving module receives the model reasoning traffic of the at least one AI model service of which the instances have been deleted first, the first instance acquires a distributed lock resource and triggers the wakeup operation, wherein the first instance is capable of preventing other instances in the plurality of instances of the traffic receiving module from triggering the wakeup operation when the first instance possesses the distributed lock resource.

8. The method of claim 1 , wherein acquiring the calling information of the at least one AI model service comprises:
acquiring the creation time of the at least one AI model service; and
acquiring at least one of the following information of the at least one AI model service by analyzing traffic logs related to all the instances of the at least one AI model service: the most recent calling time, the number of calling within a predetermined time window and calling time distribution within the predetermined time window.

9. The method of claim 1 , wherein the AI development platform is a single cluster.

10. A computing system for management of an artificial intelligence, AI, development platform, the AI development platform being deployed with instances of a plurality of AI model services, and each of the plurality of AI model services being provided with one or more instances, wherein the computing system comprises:
one or more processors; and
a non-transitory memory that stores a program, the program comprising instructions that, when executed by the one or more processors, cause the one or more processors to:
acquire calling information of all instances of at least one AI model service of the plurality of AI model services;
determine calling activity of the at least one AI model service according to the calling information; and
delete all instances of the at least one AI model service without deleting a routing configuration of the at least one AI model service in an access layer of the AI development platform in response to the determined calling activity meeting a calling frequency within the predetermined time window.

11. The computing system of claim 10 , wherein the instructions further cause the one or more processors to:
in response to determining that the at least one AI model service of which the instances have been deleted is not called within a predetermined period, delete the routing configuration of the at least one AI model service in the access layer of the AI development platform.

12. The computing system of claim 10 , wherein deleting all instances of the at least one AI model service comprises:
determining whether the at least one AI model service exists in a cluster of the AI development platform system; and
in response to determining that the at least one AI model service exists in the cluster, switching a backend of the at least one AI model service to a traffic receiving module, wherein the traffic receiving module is used for receiving a model reasoning traffic for the AI model service of which the instances have been deleted.

13. The computing system of claim 12 , wherein deleting all instances of the at least one AI model service further comprises:
continuing to store original data information of the at least one AI model service into the cluster, wherein the original data information at least comprises a first number of the instances deployed on the AI development platform before the instances of the at least one AI model service is deleted.

14. The computing system of claim 13 , wherein the instructions further cause the one or more processors to:
in response to the traffic receiving module receives the model reasoning traffic of the at least one AI model service of which the instances have been deleted, trigger a wakeup operation for at least one AI model service, wherein the wakeup operation comprises:
recovering the first number of instances of the at least one AI model service on the AI development platform, and poll states of the recovered instances until the starting of the first number of instances is completed; and
switching the backend of the at least one AI model service back to the first number of recovered instances.

15. The computing system of claim 14 , wherein in the case that the routing configuration of the at least one AI model service in the access layer of the AI development platform has b

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd

Inventors

• Yuan Zhengxiong
• Shi En
• Xie Yongkang
• Hu Mingren
• Qian Zhengyu
• Chu Zhenfang

Classifications

• H04L67/34 Primary

  involving the movement of software or configuration parameters  (network booting or remote initial program loading [RIPL] G06F9/4416) · CPC title

• G06F9/44505 Primary

  Configuring for program initiating, e.g. using registry, configuration files · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• H04L41/5054

  Automatic deployment of services triggered by the service manager, e.g. service implementation by automatic configuration of network components · CPC title

• H04L67/2895

  Intermediate processing functionally located close to the data provider application, e.g. reverse proxies · CPC title