Managing artificial intelligence model partitions for edge computing environment

Patent metadata

Field	Value
Publication number	US-2022012607-A1
Application number	US-202016925864-A
Country	US
Kind code	A1
Filing date	Jul 10, 2020
Priority date	Jul 10, 2020
Publication date	Jan 13, 2022
Grant date	—

Abstract

Techniques are disclosed for managing artificial intelligence model partitions for execution in an information processing system with edge computing resources. For example, a method comprises the following steps. An intermediate representation of an artificial intelligence model is obtained. A computation graph is generated based on the intermediate representation. The computation graph is partitioned into a set of partitions. The method then schedules the set of partitions for respective execution on a set of computing devices in an edge computing environment, and causes deployment of the set of partitions respectively to the set of computing devices for execution in the edge computing environment.

First claim

What is claimed is:

1 . An apparatus, comprising:
a processor; and a memory storing computer program instructions, the processor executing the computer program instructions in the memory to control the apparatus to: obtain an intermediate representation of an artificial intelligence model; obtain a computation graph generated based on the intermediate representation; partition the computation graph into a set of partitions; schedule the set of partitions for respective execution on a set of computing devices in an edge computing environment; and cause deployment of the set of partitions respectively to the set of computing devices for execution in the edge computing environment.

2 . The apparatus of claim 1 , wherein causing deployment of the set of partitions to the set of computing devices further comprises a model scheduler dispatching a set of partition executers for execution to the set of computing devices in the edge computing environment.

3 . The apparatus of claim 2 , wherein the model scheduler is configured as a remote procedure call server and the set of partition executers are configured as remote procedure call clients.

4 . The apparatus of claim 1 , wherein scheduling the set of partitions further comprises obtaining resource requirements for execution of the set of partitions in order to generate a scheduling plan.

5 . The apparatus of claim 1 , wherein the processor further executes the computer program instructions in the memory to control the apparatus to compile each of the set of partitions to generate a runtime instance specific to the type of computing device on which each partition is to be executed.

6 . The apparatus of claim 1 , wherein the obtaining, partitioning, scheduling and causing deployment operations are performed in accordance with a model parallelism manager layer in coordination with an application manager layer interposed between the model parallelism manager layer and the edge computing environment.

7 . The apparatus of claim 1 , wherein the processor further executes the computer program instructions in the memory to control the apparatus to re-schedule and cause re-deployment of one or more of the set of partitions when one or more of the computing devices in the edge computing environment change geographic locations.

8 . A method, comprising:
obtaining an intermediate representation of an artificial intelligence model; obtaining a computation graph generated based on the intermediate representation; partitioning the computation graph into a set of partitions; scheduling the set of partitions for respective execution on a set of computing devices in an edge computing environment; and causing deployment of the set of partitions respectively to the set of computing devices for execution in the edge computing environment; wherein the steps are performed by one or more processing devices.

9 . The method of claim 8 , wherein causing deployment of the set of partitions to the set of computing devices further comprises dispatching, via a model scheduler, a set of partition executers for execution to the set of computing devices in the edge computing environment.

10 . The method of claim 9 , wherein the model scheduler is configured as a remote procedure call server and the set of partition executers are configured as remote procedure call clients.

11 . The method of claim 8 , wherein scheduling the set of partitions further comprises obtaining resource requirements for execution of the set of partitions in order to generate a scheduling plan.

12 . The method of claim 8 , further comprising compiling each of the set of partitions to generate a runtime instance specific to the type of computing device on which each partition is to be executed.

13 . The method of claim 8 , wherein the obtaining, partitioning, scheduling and causing deployment steps are performed in accordance with a model parallelism manager layer in coordination with an application manager layer interposed between the model parallelism manager layer and the edge computing environment.

14 . The method of claim 8 , further comprising re-scheduling and causing re-deployment of one or more of the set of partitions when one or more of the computing devices in the edge computing environment change geographic locations.

15 . A computer program product stored on a non-transitory computer-readable medium and comprising machine executable instructions, the machine executable instructions, when executed, causing a processing device to:
obtain an intermediate representation of an artificial intelligence model; obtain a computation graph generated based on the intermediate representation; partition the computation graph into a set of partitions; schedule the set of partitions for respective execution on a set of computing devices in an edge computing environment; and cause deployment of the set of partitions respectively to the set of computing devices for execution in the edge computing environment.

16 . The computer program product of claim 15 , wherein causing deployment of the set of partitions to the set of computing devices further comprises a model scheduler dispatching a set of partition executers for execution to the set of computing devices in the edge computing environment, and wherein the model scheduler is configured as a remote procedure call server and the set of partition executers are configured as remote procedure call clients.

17 . The computer program product of claim 15 , wherein scheduling the set of partitions further comprises obtaining resource requirements for execution of the set of partitions in order to generate a scheduling plan.

18 . The computer program product of claim 15 , wherein the machine executable instructions, when executed, further cause a processing device to compile each of the set of partitions to generate a runtime instance specific to the type of computing device on which each partition is to be executed.

19 . The computer program product of claim 15 , wherein the obtaining, partitioning, scheduling and causing deployment steps are performed in accordance with a model parallelism manager layer in coordination with an application manager layer interposed between the model parallelism manager layer and the edge computing environment.

20 . The computer program product of claim 15 , wherein the machine executable instructions, when executed, further cause a processing device to re-schedule and cause re-deployment of one or more of the set of partitions when one or more of the computing devices in the edge computing environment change geographic locations.

Assignees

• Emc Ip Holding Co Llc

Inventors

• Liu Jinpeng
• Li Jin
• Jia Zhen
• Maclellan Christopher S

Classifications

• G06F9/5044

  considering hardware capabilities · CPC title

• G06F9/5038

  considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups) · CPC title

• G06N5/022 Primary

  Knowledge engineering; Knowledge acquisition · CPC title

• G06N20/00

  Machine learning · CPC title

• G06N3/02

  Neural networks · CPC title