Adaptive controller for online adaptation of resource allocation policies for iterative workloads using reinforcement learning

Patent metadata

Field	Value
Publication number	US-11366697-B2
Application number	US-201916400289-A
Country	US
Kind code	B2
Filing date	May 1, 2019
Priority date	May 1, 2019
Publication date	Jun 21, 2022
Grant date	Jun 21, 2022

Abstract

Techniques are provided for adaptive resource allocation for workloads. One method comprises obtaining a dynamic system model based on a relation between an amount of at least one resource for one or more iterative workloads and at least one predefined service metric; obtaining, from a resource allocation correction module, an instantaneous value of the at least one predefined service metric; and applying to a controller: (i) instantaneous parameters of the dynamic system model, and (ii) a difference between the instantaneous value of the at least one predefined service metric and a target value for the at least one predefined service metric, wherein the controller determines an adjustment to the amount of the at least one resource for the one or more iterative workloads. The obtained system model is optionally updated over time based on an amount of at least one resource added and the one or more predefined service metrics.

First claim

What is claimed is:

1. A method, comprising:
obtaining a dynamic system model that relates (i) an amount of at least one resource provided by an execution environment that executes one or more iterative workloads and (ii) at least one predefined service metric indicating a level of service provided by the execution environment for the one or more iterative workloads, wherein at least a first parameter and a second parameter of a plurality of parameters of the obtained dynamic system model are learned for a plurality of iterations of the one or more iterative workloads, wherein the learned first parameter represents an effect, on a value of the at least one predefined service metric, of an adjustment to the amount of the at least one resource provided by the execution environment and wherein the learned second parameter represents an obtained instantaneous value of the at least one predefined service metric;
and
applying to a controller: (i) the dynamic system model for a given iteration of the plurality of iterations of the one or more iterative workloads, and (ii) a difference between the instantaneous value for the given iteration of the at least one predefined service metric and a target value for the at least one predefined service metric, wherein the controller determines an adjustment, based at least in part on the difference and using the first parameter to modify one or more dynamics of a feedback loop that determines the difference, to the amount of the at least one resource to be applied in the execution environment for the one or more iterative workloads;
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.

2. The method of claim 1 , wherein the obtained dynamic system model is one or more of: (a) derived from a relation between (i) an amount of at least one resource added in the execution environment in a given iteration and (ii) the predefined service level metric and (b) predefined based at least in part on the relation between the amount of the at least one resource added in the execution environment and the predefined service level metric.

3. The method of claim 1 , wherein the obtained dynamic system model is updated over time based on an amount of at least one resource added in the execution environment in a given iteration and the one or more predefined service metrics for the given iteration.

4. The method of claim 1 , wherein the iterative workload comprises a training of a Deep Neural Network.

5. The method of claim 1 , further comprising the step of applying the adjustment to the amount of at least one resource to a saturation model that bounds the difference between the instantaneous value of the at least one predefined service metric and one or more of the minimum and maximum values for the at least one predefined service metric.

6. The method of claim 1 , further comprising the step of applying the adjustment to the amount of at least one resource to an integrator that generates a cumulative resource allocation based on the adjustment to the amount of at least one resource for the one or more iterative workloads.

7. The method of claim 6 , further comprising the step of applying the cumulative resource allocation to a saturation model that bounds an applied resource allocation within an available amount of at least one resource.

8. The method of claim 1 , wherein the at least one resource comprises one or more of a number of processing cores, an amount of memory and an amount of network bandwidth.

9. A computer program product, comprising a non-transitory machine-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed by at least one processing device perform the following steps:
obtaining a dynamic system model that relates (i) an amount of at least one resource provided by an execution environment that executes one or more iterative workloads and (ii) at least one predefined service metric indicating a level of service provided by the execution environment for the one or more iterative workloads, wherein at least a first parameter and a second parameter of a plurality of parameters of the obtained dynamic system model are learned for a plurality of iterations of the one or more iterative workloads, wherein the learned first parameter represents an effect, on a value of the at least one predefined service metric, of an adjustment to the amount of the at least one resource provided by the execution environment and wherein the learned second parameter represents an obtained instantaneous value of the at least one predefined service metric;
and
applying to a controller: (i) the dynamic system model for a given iteration of the plurality of iterations of the one or more iterative workloads, and (ii) a difference between the instantaneous value for the given iteration of the at least one predefined service metric and a target value for the at least one predefined service metric, wherein the controller determines an adjustment, based at least in part on the difference and using the first parameter to modify one or more dynamics of a feedback loop that determines the difference, to the amount of the at least one resource to be applied in the execution environment for the one or more iterative workloads.

10. The computer program product of claim 9 , wherein the obtained dynamic system model is one or more of: (a) derived from a relation between (i) an amount of at least one resource added in the execution environment in a given iteration and (ii) the predefined service level metric and (b) predefined based at least in part on the relation between the amount of the at least one resource added in the execution environment and the predefined service level metric.

11. The computer program product of claim 9 , wherein the obtained dynamic system model is updated over time based on an amount of at least one resource added in the execution environment in a given iteration and the one or more predefined service metrics for the given iteration.

12. The computer program product of claim 9 , wherein the iterative workload comprises a training of a Deep Neural Network.

13. The computer program product of claim 9 , further comprising the step of applying the adjustment to the amount of at least one resource to one or more of: (i) a saturation model that bounds the difference between the instantaneous value of the at least one predefined service metric and one or more of the minimum and maximum values for the at least one predefined service metric; and (ii) an integrator that generates a cumulative resource allocation based on the adjustment to the amount of at least one resource for the one or more iterative workloads.

14. The computer program product of claim 9 , wherein the at least one resource comprises one or more of a number of processing cores, an amount of memory and an amount of network bandwidth.

15. An apparatus, comprising:
a memory; and
at least one processing device, coupled to the memory, operative to implement the following steps:
obtaining a dynamic system model that relates (i) an amount of at least one resource provided by an execution environment that executes one or more iterative workloads and (ii) at least one predefined service metric indicating a level of service provided by the execution environment for the one or more iterative workloads, wherein at least a first parameter and a second parameter of a plurality of parameters of the obtained dynamic system model are learned for a plurality of iterations of the one or more iterative workloads, whe

Assignees

• Emc Ip Holding Co Llc

Inventors

• Calmon Tiago Salviano

Classifications

• G06F9/505

  considering the load · CPC title

• G06N3/08

  Learning methods · CPC title

• G06F9/5038 Primary

  considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups) · CPC title

• G06F9/5077

  Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455) · CPC title

• G06F9/5033

  considering data affinity · CPC title