Allocating resources in a compute farm to increase resource utilization by using a priority-based allocation layer to allocate job slots to projects

Patent metadata

Field	Value
Publication number	US-9465663-B2
Application number	US-26312608-A
Country	US
Kind code	B2
Filing date	Oct 31, 2008
Priority date	Oct 31, 2008
Publication date	Oct 11, 2016
Grant date	Oct 11, 2016

Abstract

Some embodiments provide a system for allocating resources in a compute farm. During operation, the system can receive resource-requirement information for a project. Next, the system can receive a request to execute a new job in the compute farm. In response to determining that no job slots are available for executing the new job, and that the project associated with the new job has not used up its allocated job slots, the system may execute the new job by suspending or re-queuing a job that is currently executing, and allocating the freed-up job slot to the new job. If the system receives a resource-intensive job, the system may create dummy jobs, and schedule the dummy jobs on the same computer system as the resource-intensive job to prevent the queuing system from scheduling multiple resource-intensive jobs on the same computer system.

First claim

What is claimed is:

1. A method for allocating resources in a compute farm, wherein the compute farm's resources are represented using a set of job slots, and wherein the set of job slots are managed by a queuing system, the method comprising:
a priority-based allocation (PBA) layer receiving information about a first job that is pending in the queuing system, wherein the information indicates that the first job is associated with a first project, and wherein the PBA layer is separate from the queuing system;
in response to the PBA layer determining that the number of job slots in the compute farm that are allocated to jobs associated with the first project is less than the number of job slots allocated to the first project, and that no free job slots are available for executing the first job,
the PBA layer identifying a low-priority job slot that is currently executing a second job associated with a second project, wherein the number of job slots in the compute farm that are executing jobs associated with the second project is greater than the number of job slots allocated to the second project,
the PBA layer providing a scheduling instruction to the queuing system via an interface of the queuing system, wherein the scheduling instruction instructs the queuing system to schedule the first job in the low-priority job slot, and
the PBA layer providing a de-allocation instruction to the queuing system via the interface of the queuing system, wherein the de-allocation instruction instructs the queuing system to de-allocate the executing second job.

2. The method of claim 1 , wherein the de-allocation instruction instructs the queuing system to stop and re-queue the executing second job.

3. The method of claim 1 , wherein the de-allocation instruction instructs the queuing system to suspend the executing second job.

4. A non-transitory computer-readable storage medium storing instructions for a priority-based allocation (PBA) layer that when executed by a computer cause the computer to perform a method for allocating resources in a compute farm, wherein the compute farm's resources are represented using a set of job slots, and wherein the set of job slots are managed by a queuing system, the method comprising:
the PBA layer receiving information about a first job that is pending in the queuing system, wherein the information indicates that the first job is associated with a first project, and wherein the PBA layer is separate from the queuing system;
in response to the PBA layer determining that the number of job slots in the compute farm that are allocated to jobs associated with the first project is less than the number of job slots allocated to the first project, and that no free job slots are available for executing the first job,
the PBA layer identifying a low-priority job slot that is currently executing a second job associated with a second project, wherein the number of job slots in the compute farm that are executing jobs associated with the second project is greater than the number of job slots allocated to the second project,
the PBA layer providing a scheduling instruction to the queuing system via an interface of the queuing system, wherein the scheduling instruction instructs the queuing system to schedule the first job in the low-priority job slot, and
the PBA layer providing a de-allocation instruction to the queuing system via the interface of the queuing system, wherein the de-allocation instruction instructs the queuing system to de-allocate the executing second job.

5. The computer-readable storage medium of claim 4 , wherein the de-allocation instruction instructs the queuing system to stop and re-queue the executing second job.

6. The computer-readable storage medium of claim 4 , wherein the de-allocation instruction instructs the queuing system to suspend the executing second job.

7. A computer system for allocating resources in a compute farm, wherein the compute farm's resources are represented using a set of job slots, and wherein the set of job slots are managed by a queuing system, the computer system comprising:
a processor; and
a storage medium storing instructions for a priority-based allocation (PBA) layer that, when executed by the processor, cause the computer system to perform a method, comprising:
the PBA layer receiving information about a first job that is pending in the queuing system, wherein the information indicates that the first job is associated with a first project, and wherein the PBA layer is separate from the queuing system;
in response to the PBA layer determining that the number of job slots in the compute farm that are allocated to jobs associated with the first project is less than the number of job slots allocated to the first project, and that no free job slots are available for executing the first job,
the PBA layer identifying a low-priority job slot that is currently executing a second job associated with a second project, wherein the number of job slots in the compute farm that are executing jobs associated with the second project is greater than the number of job slots allocated to the second project,
the PBA layer providing a scheduling instruction to the queuing system via an interface of the queuing system, wherein the scheduling instruction instructs the queuing system to schedule the first job in the low-priority job slot, and

the PBA layer providing a de-allocation instruction to the queuing system via the interface of the queuing system, wherein the de-allocation instruction instructs the queuing system to de-allocate the executing second job.

8. The computer system of claim 7 , wherein the de-allocation instruction instructs the queuing system to stop and re-queue the executing second job.

9. The computer system of claim 7 , wherein the de-allocation instruction instructs the queuing system to suspend the executing second job.

Assignees

• Mincarelli John R
• Synopsys Inc

Inventors

• Mincarelli John R

Classifications

• G06F2209/503

  Resource availability · CPC title

• G06F9/5027

  the resource being a machine, e.g. CPUs, Servers, Terminals · CPC title

• G06F9/5005

  to service a request · CPC title

• G06F9/485 Primary

  Task life-cycle, e.g. stopping, restarting, resuming execution (G06F9/4881 takes precedence) · CPC title

• G06F2209/5021

  Priority · CPC title