Distributed resource management method and system

Patent metadata

Field	Value
Publication number	US-10313429-B2
Application number	US-201615095822-A
Country	US
Kind code	B2
Filing date	Apr 11, 2016
Priority date	Apr 11, 2016
Publication date	Jun 4, 2019
Grant date	Jun 4, 2019

Abstract

The present disclosure is drawn to methods and systems for resource enforcement in a distributed system. At least one first host of the system has an enforcement agent configured to trigger a master enforcement controller in response to the first host running a task requiring access to a distributed resource. The master enforcement controller obtains identification information regarding the distributed resource and transmits an enforcement command to other hosts of the distributed system involved in providing the distributed resource. The enforcement command is received by enforcement agents on each of the involved hosts and triggers a slave enforcement controller on each of the involved hosts. The slave enforcement controllers locally enforce a resource quota for the distributed resource.

First claim

The invention claimed is:

1. A method for distributed resource enforcement in a distributed computing system comprising a plurality of hosts, the method comprising:
in response to an enforcement agent, provided on a first host of the plurality of hosts, running a task that requires access to a distributed resource of the distributed computing system, designating a master enforcement controller to obtain identification information regarding the distributed resource, wherein at least two of the plurality of hosts are involved in providing the distributed resource and include the first host;
transmitting, from the master enforcement controller provided on the first host, a first enforcement command including the identification information to at least one host of the at least two of the plurality of hosts involved in providing the distributed resource, the first enforcement command starting a slave enforcement controller provided on the at least one host to locally enforce a resource quota for the at least one host;
receiving, at the master enforcement controller provided on the first host, a second enforcement command from another one of the plurality of hosts of the distributed computing system, the second enforcement command including an adjusted resource quota; and
in response to receipt of the second enforcement command, designating the slave enforcement controller provided on the first host to locally enforce the adjusted resource quota for the first host.

2. The method of claim 1 , wherein the task comprises one of a High Performance Computing (HPC) batch job, a Message Passing Interface (MPI), a serial batch, real-time analytics, an elastic application, a long running service, a virtual machine, and a task container, and wherein the distributed resource comprises one of a distributed file system, a virtualized file system, a distributed database, a virtualized network, and a distributed cache.

3. The method of claim 2 , wherein obtaining identification information comprises identifying the at least one host by determining which ones of the at least two of the plurality of hosts are involved in providing the distributed resource.

4. The method of claim 1 , wherein transmitting the first enforcement command comprises transmitting the first enforcement command to two or more hosts of the at least two of the plurality of hosts.

5. The method of claim 4 , wherein the first enforcement command further comprises the resource quota for each of the two or more hosts.

6. The method of claim 5 , further comprising dynamically adjusting the resource quota based on access to the distributed resource by the first host.

7. The method of claim 6 , wherein dynamically adjusting the resource quota comprises setting different resource quotas for each of the two or more hosts.

8. A host in a distributed computing system having a plurality of hosts, the host comprising:
a processing unit; and
a non-transitory memory communicatively coupled to the processing unit and comprising computer readable program instructions executable by the processing unit for:
in response to an enforcement agent, provided on a first host of the plurality of hosts, running a task that requires access to a distributed resource of the distributed computing system, designating a master enforcement controller to obtain identification information regarding the distributed resource, wherein at least two of the plurality of hosts are involved in providing the distributed resource and include the first host;
transmitting, from the master enforcement controller provided on the first host, an enforcement command including the identification information to at least one host of the at least two of the plurality of hosts involved in providing the distributed resource, the enforcement command starting a slave enforcement controller provided on the at least one host to locally enforce a resource quota for the at least one host;
receiving, at the master enforcement controller provided on the first host, a second enforcement command from another one of the plurality of hosts of the distributed computing system, the second enforcement command including an adjusted resource quota; and
in response to receipt of the second enforcement command, designating the slave enforcement controller provided on the first host to locally enforce the adjusted resource quota for the first host.

9. The host of claim 8 , wherein the task comprises one of a High Performance Computing (HPC) batch job, a Message Passing Interface (MPI), a serial batch, real-time analytics, an elastic application, a long running service, a virtual machine, and a task container, and wherein the distributed resource comprises one of a distributed file system, a virtualized file system, a distributed database, a virtualized network, and a distributed cache.

10. The host of claim 9 , wherein obtaining identification information comprises identifying the at least one host by determining which ones of the at least two of the plurality of hosts are involved in providing the distributed resource.

11. The host of claim 8 , wherein transmitting the first enforcement command comprises transmitting the first enforcement command to two or more hosts of the at least two of the plurality of hosts.

12. The host of claim 11 , wherein the first enforcement command further comprises the resource quota for each of the two or more hosts.

13. The host of claim 12 , wherein the processing unit is further executable for dynamically adjusting the resource quota based on access to the distributed resource by the first host.

14. The host of claim 13 , wherein dynamically adjusting the resource quota comprises setting different resource quotas for each of the two or more hosts.

15. A distributed computing system comprising a plurality of hosts, at least two of the plurality of hosts are involved in providing a distributed resource, at least one first host of the plurality of hosts running a task that requires access to the distributed resource and comprising an enforcement agent, the at least two of the plurality of hosts involved in providing the distributed resource including the at least one first host, the enforcement agent configured for locally designating a master enforcement controller on the at least one first host to obtain identification information regarding the distributed resource in response to the enforcement agent requiring access to the distributed resource and transmitting a first enforcement command with the identification information to the at least one host of the two of the plurality of hosts involved in providing the distributed resource, the first enforcement command designating a slave enforcement controller provided on the at least one host upon receipt of a second enforcement command including an adjusted resource quota from another one of the plurality of hosts to locally enforce the adjusted resource quota on the at least one host.

16. The distributed computing system of claim 15 , wherein each one of the plurality of hosts comprise an enforcement agent configured for locally designating the master enforcement controller and the slave enforcement controller.

Assignees

• Huawei Tech Co Ltd

Inventors

• CHEN Jin
• Guo Lei
• Chen Chong

Classifications

• H04L67/10 Primary

  in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) · CPC title

• H04L67/60 Primary

  Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources (admission control or resource allocation H04L47/70) · CPC title