Job distribution within a grid environment using clusters of execution hosts

Patent metadata

Field	Value
Publication number	US-11243805-B2
Application number	US-201916363503-A
Country	US
Kind code	B2
Filing date	Mar 25, 2019
Priority date	Jun 20, 2012
Publication date	Feb 8, 2022
Grant date	Feb 8, 2022

Abstract

A technique for job distribution within a grid environment includes receiving jobs at a submission cluster for distribution of the jobs to at least one of a plurality of execution clusters where each execution cluster includes one or more execution hosts. Resource attributes are determined corresponding to each execution host of the execution clusters. Resource requirements are determined for the job and candidate execution clusters are identified for processing the job based on the resource attributes of the execution hosts and the resource requirements of the job. An optimum execution cluster is selected from the candidate execution clusters for allocating the job thereto for execution of the job based on a weighting factor applied to select resources of the respective execution clusters.

First claim

What is claimed is:

1. A method for job distribution within a grid environment, comprising:
receiving two or more jobs of a plurality of submitted jobs at a submission cluster for distribution of the two or more jobs to at least one of a plurality of execution clusters, each execution cluster having one or more execution hosts;
determining resource requirements for each of the two or more jobs;
determining resource attributes corresponding to each execution host of the execution cluster using a processor unit that evaluates computing resources of the execution hosts, wherein the computing resources are needed to execute the job according to the resource requirements;
identifying candidate execution clusters for processing the two or more jobs based on:
the existence of the computing resources of the execution hosts and the resource requirements of the job; and
the availability of the computing resources of the execution hosts;
selecting an optimum execution cluster from the candidate execution clusters for allocating each of the two or more jobs thereto for execution of the two or more jobs based on a weighting factor applied to select computing resources of the respective execution clusters, the weighting factor including a value assigned to the select computing resources representing an importance level of the select computing resources relative to other computing resources required for the job;
grouping the two or more jobs based on job resource requirements;
grouping, for each execution cluster, execution hosts based on the resource attributes of the respective execution hosts;
defining, for each grouping of execution hosts, a mega-host for the respective execution cluster, the mega-host for a respective execution cluster defining resource attributes based on the resource attributes of the respective grouped execution hosts;
creating a plurality of priority queues based on priorities of the grouped two or more jobs;
assigning each group of jobs to a corresponding priority queue; and
identifying candidate mega-hosts for the submitted jobs based on the resource attributes of the respective mega-hosts and the resource requirements of the group of jobs.

2. The method of claim 1 , further comprising grouping the execution hosts for a respective execution cluster based on select resource attributes for the respective execution hosts.

3. The method of claim 1 , further comprising grouping the execution hosts for a respective execution cluster based on resource slots and memory capacity attributes for the respective execution hosts.

4. The method of claim 1 , further comprising sorting the candidate execution clusters based on available resources of the respective candidate execution clusters.

5. The method of claim 1 , further comprising applying a job forwarding policy for each of the two or more jobs based on a division of the resource attributes of the execution clusters among users requesting job processing.

6. The method of claim 1 , further comprising:
determining a number of running jobs at the candidate execution clusters; and
determining a number of forwarded pending jobs at the candidate execution clusters forwarded from the submission cluster.

7. A system for job distribution within a grid environment, comprising:
a submission cluster, having a processor unit, for distributing two or more jobs of a plurality of jobs to at least one of a plurality of execution clusters, wherein each execution cluster comprises one or more execution hosts, and wherein the submission cluster comprises logic executable by the processor unit to:
determine resource requirements for the two or more jobs;
determine resource attributes corresponding to each execution host of the execution cluster using the processor unit that evaluates computing resources of the execution hosts, wherein the computing resources are needed to execute the two or more jobs according to the resource requirements;
identify candidate execution clusters for processing the two or more jobs based on:
the existence of the computing resources of the execution hosts and the resource requirements of the job; and
based on the existence of the computing resources of the execution hosts and the resource requirements of the job, the availability of the computing resources of the execution hosts; and
select an optimum execution cluster from the candidate execution clusters for allocating each of the two or more jobs thereto for execution of the two or more jobs based on a weighting factor applied to select computing resources of the respective execution clusters, the weighting factor including a value assigned to the select computing resources representing an importance level of the select computing resources relative to other computing resources required for the two or more jobs;
group the two or more jobs based on job resource requirements;
group, for each execution cluster, execution hosts based on the resource attributes of the respective execution hosts;
define, for each grouping of execution hosts, a mega-host for the respective execution cluster, the mega-host for a respective execution cluster defining resource attributes based on the resource attributes of the respective grouped execution hosts;
create a plurality of priority queues based on priorities of the grouped two or more jobs;
assign each group of jobs to a corresponding priority queue; and
identify candidate mega-hosts for the submitted jobs based on the resource attributes of the respective mega-hosts and the resource requirements of the group of jobs.

8. The system of claim 7 , wherein the logic is executable to group the execution hosts for a respective execution cluster based on select resource attributes for the respective execution hosts.

9. The system of claim 7 , wherein the logic is executable to group the execution hosts for a respective execution cluster based on resource slots and memory capacity attributes for the respective execution hosts.

10. The system of claim 7 , wherein the logic is executable to sort the candidate execution clusters based on available resources of the respective candidate execution clusters.

11. The system of claim 7 , wherein the logic is executable to apply a job forwarding policy for each of the two or more jobs based on a division of the resource attributes of the execution clusters among users requesting job processing.

12. The system of claim 7 , wherein the logic is executable to:
determine a number of running jobs at the candidate execution clusters; and
determine a number of forwarded pending jobs at the candidate execution clusters forwarded from the submission cluster.

13. A computer program product for job distribution within a grid environment, the computer program product comprising:
a non-transitory computer readable medium having computer readable program code embodied therewith, the computer readable program code comprising computer readable program code executable by a processor to:
determine resource requirements for a plurality of submitted jobs;
determine resource attributes corresponding to each execution host of a plurality of execution clusters using the processor unit that evaluates computing resources of the execution hosts, wherein the computing resources are needed to execute the jobs according to the resource requirements;
identify candidate execution clusters for processing each job of the plurality of submitted jobs based on:
the existence of the computing resources of the execution hosts and the resource requirements of each job; and
based on the existence of the computing resources of the execution hosts and the resource requiremen

Assignees

• IBM

Inventors

• Chen Chong
• Liu Fang
• Wang Qi
• Yuan Shutao

Classifications

• G06F9/5072 Primary

  Grid computing · CPC title

• H04L67/1001

  for accessing one among a plurality of replicated servers · CPC title

• G06F9/5061

  Partitioning or combining of resources · CPC title

• G06F2209/5012

  Processor sets · CPC title

• G06F9/505

  considering the load · CPC title