Task allocation among devices in a distributed data storage system

Patent metadata

Field	Value
Publication number	US-11240305-B2
Application number	US-201615222729-A
Country	US
Kind code	B2
Filing date	Jul 28, 2016
Priority date	Jul 28, 2016
Publication date	Feb 1, 2022
Grant date	Feb 1, 2022

Abstract

In one example, a processor may receive a first request to process a first task, the first request including a first estimated central processing unit utilization for the first task and a first estimated memory utilization for the first task and receive central processing unit capacities and memory capacities of a plurality of sub-data routers including at least a first sub-data router. The processor may further determine that the first sub-data router has a lowest central processing unit capacity from among the plurality of sub-data routers that is sufficient to accommodate the first estimated central processing unit utilization for the first task and determine that the first sub-data router has a memory capacity that is sufficient to accommodate the first estimated memory utilization for the first task. The processor may then assign the first task to the first sub-data router.

First claim

What is claimed is:

1. A device comprising:
a processor; and
a computer-readable medium storing instructions which, when executed by the processor, cause the processor to perform operations of a first task handler, the operations comprising:
receiving a first request for processing of a first task, the first request including a first estimated central processing unit utilization for executing the first task on at least one sub-data router of a plurality of sub-data routers and a first estimated memory utilization for executing the first task on the at least one sub-data router, the first task, when executed on the at least one sub-data router of the plurality of sub-data routers, comprising the at least one sub-data router accessing at least a portion of a file stored in a distributed data storage system;
periodically receiving a respective central processing unit capacity and a respective memory capacity of each of the plurality of sub-data routers, the plurality of sub-data routers including at least the first sub-data router;
determining that a first sub-data router has a lowest central processing unit capacity from among the plurality of sub-data routers that is sufficient to accommodate the first estimated central processing unit utilization for the first task;
determining that the first sub-data router has a memory capacity that is sufficient to accommodate the first estimated memory utilization for the first task, the first estimated memory utilization comprising an estimated quantity of memory that at least one central processing unit of the first-sub data router is to use for at least instructions and data for the processing of the first task, the processing of the first task comprising the first sub-data router identifying at least one data node having the at least the portion of the file stored in the distributed data storage system and the first sub-data router sending instructions to the at least one data node of the distributed data storage system that is identified to perform at least one function associated with the accessing of the at least the portion of the file stored in the distributed data storage system in accordance with the first task; and
sending the first task to the first sub-data router based on the determining that the first sub-data router has the lowest central processing unit capacity that is sufficient and the determining that the first sub-data router has the memory capacity that is sufficient for the processing of the first task, the processing of the first task comprising the first sub-data router identifying at least one data node having the at least the portion of the file stored in the distributed data storage system and the first sub-data router sending instructions to the at least one data node of the distributed data storage system that is identified to perform at least one function associated with the accessing of the at least the portion of the file stored in the distributed data storage system in accordance with the first task.

2. The device of claim 1 , wherein the operations further comprise:
receiving a second request to process a second task, the second request including a second estimated central processing unit utilization for the second task and a second estimated memory utilization for the second task.

3. The device of claim 2 , wherein the operations further comprise:
determining, in accordance with the sending of the first task to the first sub-data router, that a second sub-data router of the plurality of sub-data routers has a current lowest central processing unit capacity from among the plurality of sub-data routers that is sufficient to accommodate the second estimated central processing unit utilization for the second task;
determining that the second sub-data router has a memory capacity that is sufficient to accommodate the second estimated memory utilization for the second task; and
sending the second task to the second sub-data router based on the determining that the second sub-data router has the current lowest central processing unit capacity that is sufficient and the determining that the second sub-data router has the memory capacity that is sufficient.

4. The device of claim 2 , wherein the operations further comprise:
determining that the second estimated central processing unit utilization for the second task is below a threshold central processing unit utilization;
determining that the second estimated memory utilization for the second task is below a threshold memory utilization; and
sending the second task to a sub-data router with a lowest latency from among the plurality of sub-data routers.

5. The device of claim 1 , wherein the first request further includes a first latency requirement of the first task, wherein the first task is sent to the first sub-data router when it is determined that: the first sub-data router has the lowest central processing unit capacity that is sufficient to accommodate the first estimated central processing unit utilization for the first task, the first sub-data router has the memory capacity that is sufficient to accommodate the first estimated memory utilization for the first task, and a latency between the device and the first sub-data router is in accordance with the first latency requirement.

6. The device of claim 1 , wherein the device comprises the first task handler, and wherein the sending of the first task to the first sub-data router is in accordance with a priority of assignment between the first task handler and a second task handler.

7. The device of claim 6 , wherein the priority of assignment is based upon pre-designated ranks that are provided for the first task handler and for the second task handler.

8. The device of claim 6 , wherein the priority of assignment is in accordance with respective times since a last sending of a task by the first task handler and a last sending of a task by the second task handler.

9. The device of claim 1 , wherein the operations further comprise:
receiving a notification from the first sub-data router that the first task is placed in a processing queue at the first sub-data router.

10. The device of claim 9 , wherein the operations further comprise:
sending the first task to a second sub-data router of the plurality of sub-data routers; and
transmitting a cancellation regarding the first task to the first sub-data router.

11. The device of claim 1 , wherein the device comprises the first task handler for sending tasks to the plurality of sub-data routers, and wherein the sending the first task to the first sub-data router comprises:
sending a notification to a second task handler of the sending of the first task to the first sub-data router; and
sending the first task to the first sub-data router after a hold-off time.

12. The device of claim 11 , wherein the hold-off time is in accordance with a latency between the first task handler and the second task handler.

13. The device of claim 1 , wherein each of the plurality of sub-data routers includes a list of blocks of files that are stored in data nodes of the distributed data storage system.

14. The device of claim 1 , wherein the device comprises the first task handler for sending tasks from a first plurality of requesting devices to the plurality of sub-data routers, wherein at least a second task handler is for sending tasks from a second plurality of requesting devices to the plurality of sub-data routers.

15. The device of claim 1 , wherein the operations further comprise:
receiving, from the first sub-data router, an actual central processing unit utilization and an actual memory utili

Assignees

• At & T Ip I Lp

Inventors

• Meredith Sheldon Kent
• Cottrill William
• Zerick Juliette

Classifications

• H04L67/60

  Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources (admission control or resource allocation H04L47/70) · CPC title

• H04L67/1097 Primary

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F9/5044

  considering hardware capabilities · CPC title

• G06F9/5027

  the resource being a machine, e.g. CPUs, Servers, Terminals · CPC title

• H04L67/32

  Electricity · mapped topic