System and method of providing system jobs within a compute environment

Patent metadata

Field	Value
Publication number	US-11537434-B2
Application number	US-202217700767-A
Country	US
Kind code	B2
Filing date	Mar 22, 2022
Priority date	Nov 8, 2004
Publication date	Dec 27, 2022
Grant date	Dec 27, 2022

Abstract

The disclosure relates to systems, methods and computer-readable media for using system jobs for performing actions outside the constraints of batch compute jobs submitted to a compute environment such as a cluster or a grid. The method for modifying a compute environment from a system job disclosure associating a system job to a queuable object, triggering the system job based on an event and performing arbitrary actions on resources outside of compute nodes in the compute environment. The queuable objects include objects such as batch compute jobs or job reservations. The events that trigger the system job may be time driven, such as ten minutes prior to completion of the batch compute job, or dependent on other actions associated with other system jobs. The system jobs may be utilized also to perform rolling maintenance on a node by node basis.

First claim

What is claimed is:

1. A non-transitory computer-readable medium storing instructions for managing a multi-node compute environment having a computerized scheduler process associated therewith, the instructions when executed by a processor of a computerized device, performs operations comprising:
receiving a submission of at least one workload to be processed by the multi-node compute environment, the submission also comprising data indicating a particular quality of service (QoS) associated with the processing of the at least one workload;
based at least on the submission, creating least one software process, the at least one software process associated with the at least one workload;
identifying at least one resource necessary for processing of the at least one workload according to the particular QoS;
determining that the at least one resource has been made available for processing of the at least one workload; and
performing via the at least one software process and based on the determination, of one or more configuration actions that are outside of a compute space allocated by the computerized scheduler process,
wherein:
the performance of the one or more configuration actions that are outside the compute space allocated by the computerized scheduler process comprises performance of at least one of (i) a data stage-in operation, or (ii) a data stage-out operation; and
the instructions are further configured to, when executed, cause execution of at least one of the at least one workload according to the QoS using at least one of (i) data staged-in via the data stage-in operation, or (ii) data staged-out via the data stage-out operation, respectively.

2. The non-transitory computer-readable medium of claim 1 , wherein the at least one software process comprises an event policy relating to occurrence of one or more events, and completion of the of one or more configuration actions comprises the occurrence of the one or more events.

3. The non-transitory computer-readable medium of claim 1 , wherein the at least one software process comprises an event policy relating to occurrence of one or more events, and the determination that the at least one resource has been made available for processing of the at least one workload comprises the occurrence of the one or more events.

4. The non-transitory computer-readable medium of claim 1 , wherein the one or more configuration actions comprise one or more provisioning actions related to the at least one resource.

5. The non-transitory computer-readable medium of claim 1 , wherein the data indicating a particular quality of service (QoS) associated with the processing of the at least one workload comprises data indicative of a requirement for at least one dedicated resource.

6. The non-transitory computer-readable medium of claim 1 , wherein:
the at least one software process comprises an event policy relating to occurrence of one or more events, and the completion of the one or more configuration actions comprises the occurrence of the one or more events;
the one or more configuration actions comprise one or more provisioning actions related to the at least one resource; the data indicating a particular quality of service (QoS) associated with the processing of the at least one workload comprises data indicative of a requirement for at least one dedicated resource; and
the at least one dedicated resource comprises the at least one resource.

7. The non-transitory computer-readable medium of claim 1 , wherein:
the at least one software process comprises a system job; and
at least the creation of the system job is performed by the computerized scheduler process.

8. The non-transitory computer-readable medium of claim 1 , wherein:
the at least one software process comprises a system job; and
the association of the system job with the at least one workload comprises at least an encapsulation of at least one of (i) at least a portion of the submission, or (ii) at least a portion of the at least one workload, within the system job.

9. The non-transitory computer-readable medium of claim 1 , wherein:
the at least one software process comprises a system job; and
the performance of the one or more configuration actions that are outside of the compute space allocated by the computerized scheduler process comprises the performance of a plurality of steps having at least one temporal time constraint relating to performance or completion of at least one of the plurality of steps.

10. The non-transitory computer-readable medium of claim 1 , wherein:
the at least one software process comprises at least one queuable object; and
the at least one queueable object is queued for processing by the computerized scheduler process.

11. A non-transitory computer-readable medium storing instructions for managing a multi-node compute environment having a computerized scheduler process associated therewith, the instructions when executed by a processor of a computerized device, performs operations comprising:
receiving a submission of at least one workload to be processed by the multi-node compute environment, the submission requiring a particular quality of service (QoS) for the processing of the at least one workload;
based at least on the submission, creating at least one software process, the at least one software process associated with the at least one workload;
configuring, via the at least one software process, of at least one resource necessary for processing of the at least one workload according to the particular QoS; and
based at least on completion of the configuring of the at least one resource, performing via the at least one software process, of at least part of the processing of the at least one workload;
wherein the configuring comprises at least one of (i) staging data in, or (ii) staging data out; and
wherein the performing via the at least one software process, of the at least part of the processing of the at least one workload, comprises execution of at least part of the at least one workload according to the QoS and using the at least one of the (i) the staged-in data, or (ii) the staged-out data.

12. The non-transitory computer-readable medium of claim 11 , wherein the configuring comprises one or more actions not able to be performed by the computerized scheduler process.

13. The non-transitory computer-readable medium of claim 11 , wherein the at least one software process comprises an event policy relating to occurrence of one or more events, and completion of the configuration configuring of the at least one resource comprises the occurrence of the one or more events.

14. The non-transitory computer-readable medium of claim 11 , wherein the at least one software process comprises an event policy relating to occurrence of one or more events, and a determination that the at least one resource has been made available for processing of the at least one workload comprises the occurrence of the one or more events.

15. The non-transitory computer-readable medium of claim 11 , wherein the configuring comprises provisioning of data storage resources associated with one or more nodes of the multi-node compute environment.

16. The non-transitory computer-readable medium of claim 15 , wherein the provisioning of data storage resources associated with one or more nodes of the multi-node compute environment comprises provisioning of data resources in data communication with one or more nodes to which the at least one of the at least one workload has been allocated by the computerized scheduler process.

17. The non-transito

Assignees

• Iii Holdings 12 Llc

Inventors

• Jackson David B

Classifications

• G06F8/61

  Installation · CPC title

• G06F9/5011 Primary

  the resources being hardware resources other than CPUs, Servers and Terminals · CPC title

• G06F9/542

  Event management; Broadcasting; Multicasting; Notifications · CPC title

• G06F9/4843 Primary

  by program, e.g. task dispatcher, supervisor, operating system · CPC title