Resource comparison based task scheduling method, apparatus, and device

Patent metadata

Field	Value
Publication number	US-10216542-B2
Application number	US-201415127001-A
Country	US
Kind code	B2
Filing date	Mar 17, 2014
Priority date	Mar 17, 2014
Publication date	Feb 26, 2019
Grant date	Feb 26, 2019

Abstract

Embodiments of the present invention disclose a task scheduling method, apparatus, and device, where the method includes: acquiring information about an available computing resource used for task execution; when a configured task set is received, determining resource consumption information of each task in the task set; and comparing the resource consumption information of each task in the task set with the information about the available computing resource, and determining a task that is in the task set and whose corresponding comparison result meets a preset task scheduling condition as a target task. In addition, a generated task scheduling diagram includes information such as tasks, dependence between the tasks, resource consumption information, and execution progress of a target task. By using the present invention, task scheduling can be optimized, so that task scheduling optimization is more intuitive, thereby improving processing performance of a data warehouse.

First claim

What is claimed is:

1. A task scheduling method for scheduling tasks to be executed by a data warehouse system, the method comprising:
scheduling, at a task scheduling system, a set of configured tasks to be executed by the data warehouse system, wherein scheduling the set of configured tasks includes:
managing a preset task scheduling condition;
acquiring real-time available resource information about one or more computing resources available for task execution in the data warehouse system;
receiving, from a task deployment system, instructions to schedule the set of configured tasks;
determining resource consumption information regarding each configured task in the set of the configured tasks;
comparing the resource consumption information regarding each configured task in the set with the available resource information to obtain a comparison result for the configured task; and
identifying a target task from the set of configured tasks by virtue of the target task having a corresponding comparison result that meets the preset task scheduling condition, the preset task scheduling condition specifying consumption resource regarding the target task is less than the one or more available computing resources in the data warehouse system; and

delivering, at the task scheduling system, the target task to the task deployment system for the task deployment system to deploy the target task on the data warehouse for execution; and, wherein
comparing the resource consumption information of each configured task with the available resource information comprises:
determining, from the set according to a task cluster type indicated by the resource consumption information of each configured task in set and an available-cluster type indicated by the information about the computing resource available for task execution, a task subset whose task cluster type matches the available-cluster type;
comparing a resource consumption amount indicated by resource consumption information of a configured task in the task subset with an available-resource amount indicated by the information about the computing resource available for task execution; and
when a comparison result indicates that the resource consumption amount of the task is less than the available-resource amount, recording that the comparison result corresponding to the task meets the preset task scheduling condition; and, wherein identifying the target task from the set of the configured tasks comprises:
using at least one task in the task subset having a recorded comparison result that meets the task scheduling condition as a target task in the current scheduling period.

2. The method according to claim 1 , further comprising:
generating, at the task scheduling system, a task scheduling diagram according to task dependence and/or time dependence of each configured task in the set, wherein
the task scheduling diagram comprises a task tag that identifies each configured task in the set and a relationship tag that identifies the task dependence and/or the time dependence between the configured tasks in the set.

3. The method according to claim 2 , wherein the task scheduling diagram further comprises a resource consumption information tag of each configured task in the set and execution progress of the target task determined from the set, wherein
the resource consumption information tag identifies the resource consumption information of each configured task in the set, and the execution progress of the target task is obtained according to resource consumption information of the target task and currently detected information about a resource that has been consumed in a process of executing the target task.

4. The method according to claim 1 , wherein determining resource consumption information regarding each configured task in the set comprises:
searching a task resource consumption mapping table for actual resource consumption information, in a previous scheduling period, of each configured task in the set; and
estimating resource consumption information, in a current scheduling period, of each configured task in the set according to the actual resource consumption information in the previous scheduling period;
wherein the task resource consumption mapping table is preset and records the actual resource consumption information, in the previous scheduling period, that is obtained after each configured task in the set is executed in the previous scheduling period.

5. The method according to claim 1 , wherein comparing the resource consumption information of each configured task in the set with the available resource information comprises:
comparing a resource consumption amount indicated by the resource consumption information of each configured task in the set with an available-resource amount indicated by the information about the computing resource available for task execution; and
when a comparison result indicates that the resource consumption amount of the configured task is less than the available-resource amount, recording that the comparison result corresponding to the configured task meets the preset task scheduling condition; and, wherein identifying the target task from the set of configured tasks comprises:
identifying at least one configured task in the set having a recorded comparison result that meets the task scheduling condition as a target task in the current scheduling period.

6. The method according to claim 1 , after identifying the target configured task from the set of the configured tasks, the method further comprising:
controlling execution of the target task determined from the set;
after the execution of the target task is completed, acquiring actual resource consumption information, in the current scheduling period, of the target task; and
updating, according to the actual resource consumption information in the current scheduling period, actual resource consumption information that is of the target task and is recorded in the preset task resource consumption mapping table.

7. The method according to claim 3 , further comprising:
when a resource consumption amount indicated by the resource consumption information of each configured task in set is greater than an available-resource amount indicated by the information about the computing resource available for task execution, sending, at the task scheduling system, a prompt message to prompt for optimization of the configured task; and
when task execution progress generated in the task scheduling diagram is less than a preset threshold, sending, at the task scheduling system, a prompt message to prompt for optimization of the configured task.

8. A computer storage medium, wherein:
the computer storage medium may store a program, and when the program is executed, steps described in claim 1 are performed.

9. A task scheduling device, comprising a processor and a communications interface, wherein
the communications interface is configured to receive a set of configured tasks and available resource information regarding one or more computing resources in a data warehouse available for task execution; and
the processor is configured to:
schedule, at a task scheduling system, a set of configured tasks to be executed by the data warehouse system, wherein scheduling the set of configured tasks includes:
managing a preset task scheduling condition;
acquiring real-time available resource information about one or more computing resources available for task execution in the data warehouse system;
receiving, from a task deployment system, instructions to schedule the set of configured tasks;
determining resource consumption information regarding each configured task in the set of the configured ta

Assignees

• Huawei Tech Co Ltd

Inventors

• Wang Guangming
• Zhou Hongbing

Classifications

• G06F9/50 Primary

  Allocation of resources, e.g. of the central processing unit [CPU] · CPC title

• G06F9/4881

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title

• G06F9/5005 Primary

  to service a request · CPC title