Method, device, and computer program product for managing machine learning model

Patent metadata

Field	Value
Publication number	US-12093553-B2
Application number	US-202117320392-A
Country	US
Kind code	B2
Filing date	May 14, 2021
Priority date	Apr 23, 2021
Publication date	Sep 17, 2024
Grant date	Sep 17, 2024

Abstract

A method includes determining a first instance of a current version for a machine learning model and a second instance of an upgraded version for the machine learning model, the first instance executing a service for processing data; adjusting respectively, if it is determined that the service is to be migrated from the first instance to the second instance, a first allocation policy for storage space of the first instance and a second allocation policy for storage space of the second instance to a first target policy and a second target policy, wherein the first target policy is used to phase out storage space and the second target policy is used to phase in storage space; reclaiming allocated storage space for the first instance based on the first target policy; and allocating required storage space for the second instance based on the second target policy to realize migration of the service.

First claim

What is claimed is:

1. A method for managing a machine learning model, comprising:
determining a first instance of a current version for the machine learning model and a second instance of an upgraded version for the machine learning model, the first instance executing a service for processing data, wherein the first instance and the second instance are configured to run at least in part concurrently with one another on one or more graphics processing units to provide uninterrupted access to the service for processing data using one of the first instance and the second instance in conjunction with migration of the service from the first instance to the second instance;
adjusting respectively, if determining that the service is to be migrated from the first instance to the second instance, a first allocation policy for storage space of the first instance and a second allocation policy for storage space of the second instance to a first target policy and a second target policy, wherein the first target policy is used to phase out storage space and the second target policy is used to phase in storage space;
reclaiming allocated storage space for the first instance based on the first target policy; and
allocating required storage space for the second instance based on the second target policy to realize migration of the service;
wherein the storage space comprises memory resources of the one or more graphics processing units;
wherein the first target policy is more restrictive with regard to usage of the memory resources of the one or more graphics processing units than the first allocation policy; and
wherein the second target policy is less restrictive with regard to usage of the memory resources of the one or more graphics processing units than the first target policy.

2. The method according to claim 1 , wherein adjusting the first allocation policy and the second allocation policy to the first target policy and the second target policy respectively comprises:
determining a first allocation policy for the first instance and a second allocation policy for the second instance;
adjusting the first allocation policy of the first instance as the first target policy; and
adjusting the second allocation policy of the second instance as the second target policy.

3. The method according to claim 1 , further comprising:
if determining that the migration is completed,
replacing the first target policy with the first allocation policy for use by the first instance; and
replacing the second target policy with the second allocation policy for use by the second instance.

4. The method according to claim 1 , wherein reclaiming allocated storage space for the first instance comprises:
determining a sequence of nodes corresponding to operations associated with the machine learning model, the nodes being sorted according to an order of the corresponding operations;
determining a current node corresponding to an operation being executed in the first instance;
determining, from the sequence, a previous node previous to the current node; and
reclaiming storage space for the operation corresponding to the previous node in the first instance.

5. The method according to claim 4 , wherein reclaiming allocated storage space for the first instance further comprises:
reclaiming, after completing calculation of the current node, the storage space for the operation corresponding to the current node in the first instance.

6. The method according to claim 1 , wherein allocating required storage space for the second instance comprises:
allocating, if determining that the second allocation policy for the second instance is a policy for allocating storage space according to needs of an instance, the required storage space to the second instance by using the second allocation policy as the second target policy; and
determining, if determining that the second allocation policy for the second instance is a policy for allocating storage space of a predetermined size to an instance, allocation of the required storage space based on the allocated storage space for the second instance and the required storage space.

7. The method according to claim 6 , wherein determining allocation of the required storage space comprises:
allocating, if determining that the sum of the size of the allocated storage space and the size of the required storage space is smaller than or equal to the predetermined size, the required storage space to the second instance; and
releasing, if determining that the sum of the size of the allocated storage space and the size of the required storage space is larger than the predetermined size, at least part of the allocated storage space for the required storage space.

8. An electronic device, comprising:
at least one processing unit; and
at least one memory coupled to the at least one processing unit and storing machine-executable instructions that, when executed by the at least one processing unit, cause the electronic device to perform actions, the actions comprising:
determining a first instance of a current version for a machine learning model and a second instance of an upgraded version for the machine learning model, the first instance executing a service for processing data, wherein the first instance and the second instance are configured to run at least in part concurrently with one another on one or more graphics processing units to provide uninterrupted access to the service for processing data using one of the first instance and the second instance in conjunction with migration of the service from the first instance to the second instance;
adjusting respectively, if determining that the service is to be migrated from the first instance to the second instance, a first allocation policy for storage space of the first instance and a second allocation policy for storage space of the second instance to a first target policy and a second target policy, wherein the first target policy is used to phase out storage space and the second target policy is used to phase in storage space;
reclaiming allocated storage space for the first instance based on the first target policy; and
allocating required storage space for the second instance based on the second target policy to realize migration of the service;
wherein the storage space comprises memory resources of the one or more graphics processing units;
wherein the first target policy is more restrictive with regard to usage of the memory resources of the one or more graphics processing units than the first allocation policy; and
wherein the second target policy is less restrictive with regard to usage of the memory resources of the one or more graphics processing units than the first target policy.

9. The electronic device according to claim 8 , wherein adjusting the first allocation policy and the second allocation policy to the first target policy and the second target policy respectively comprises:
determining a first allocation policy for the first instance and a second allocation policy for the second instance;
adjusting the first allocation policy of the first instance as the first target policy; and
adjusting the second allocation policy of the second instance as the second target policy.

10. The electronic device according to claim 8 , wherein the actions further comprise:
if determining that the migration is completed,
replacing the first target policy with the first allocation policy for use by the first instance; and
replacing the second target policy with the second allocation policy for use by the second instance.

11. The electronic device according to claim 8 , wherein reclaiming allocated storage space

Assignees

• Emc Ip Holding Co Llc

Inventors

• Liu Jinpeng
• Ni Jiacheng
• Chen Qiang
• Sha Danqing
• Jia Zhen

Classifications

• G06F3/0679

  Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP] · CPC title

• G06F12/0862

  with prefetch · CPC title

• G06F12/0261

  using reference counting · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06Q10/06316

  Sequencing of tasks or work · CPC title