Allocating physical nodes for processes in an execution plan

Patent metadata

Field	Value
Publication number	US-10333800-B2
Application number	US-201715648368-A
Country	US
Kind code	B2
Filing date	Jul 12, 2017
Priority date	Sep 29, 2014
Publication date	Jun 25, 2019
Grant date	Jun 25, 2019

Abstract

Provided are a computer program product, system, and method for allocating physical nodes for processes in an execution plan. An execution plan is generated indicating a plurality of processes. A resource requirement is generated indicating requested physical nodes and an assignment of the processes to execute on the requested physical nodes. A determination is made from the resource requirement of a resource allocation of physical nodes for the requested physical nodes and the processes. The execution plan is updated to generate an updated execution plan indicating the physical nodes on which the processes will execute according to the received resource allocation.

First claim

What is claimed is:

1. A computer program product for obtaining resources for an execution plan, comprising a computer readable storage medium having computer readable program code embodied therein that when executed performs operations, the operations comprising:
generating an execution plan indicating a plurality of processes;
generating a resource requirement indicating requested physical nodes of a plurality of physical nodes available to execute processes, and an assignment of the processes to execute on the requested physical nodes;
determining physical nodes having sufficient resources for the processes assigned to the requested physical nodes;
returning a resource allocation of the determined physical nodes having the sufficient resources for the processes assigned to the requested physical nodes; and
updating the execution plan to generate an updated execution plan indicating the determined physical nodes for the requested physical nodes on which the processes assigned to the requested physical nodes will execute according to the resource allocation.

2. The computer program product of claim 1 , wherein the determining the physical nodes performs for each requested physical node of the requested physical nodes:
determining whether there is one physical node having sufficient resources for at least one process allocated to the requested physical node;
allocating the determined one physical node to the at least one process assigned to the requested physical node in response to determining that there is one physical node having the sufficient resources; and
allocating multiple physical nodes for the at least one process allocated to the requested physical node when there is not one physical node having sufficient resources for the processes assigned to the requested physical node.

3. The computer program product of claim 2 , wherein the allocating multiple physical nodes for one of the requested physical nodes comprises allocating a minimum number of physical nodes having sufficient resources for the processes that are assigned to the requested physical node.

4. The computer program product of claim 2 , wherein the processes include section leaders and players, wherein each of the players are assigned to one of a plurality of section leaders, wherein the resource requirement assigns each of the section leaders and the players assigned to the section leader to one of the requested physical nodes to prefer an allocation of each of the section leaders and the players assigned to the section leader to a separate of the physical nodes.

5. The computer program product of claim 1 , wherein the operations further comprise:
generating a configuration file template providing placeholder names for the requested physical nodes, wherein the generated execution plan assigns the processes to execute on the placeholder names for the requested physical nodes.

6. The computer program product of claim 1 , wherein the execution plan is for an extract, transform, and load (“ETL”) program, wherein the processes include a conductor, section leaders, and players assigned to the section leaders, wherein the conductor manages execution of the section leaders and the players, and wherein the section leaders manage execution of players assigned to the section leaders.

7. The computer program product of claim 1 , wherein the physical nodes assigned differ in quantity from a number of the requested physical nodes in the resource requirement.

8. The computer program product of claim 1 , wherein the allocated physical nodes provide multiple physical nodes for one of the requested physical nodes, and wherein the processes assigned to a requested physical node for which physical nodes were allocated, execute on the multiple physical nodes.

9. A system for obtaining resources for an execution plan, comprising
a processor; and
a computer readable storage medium having computer readable program code embodied therein that when executed by the processor performs operations, the operations comprising:
generating an execution plan indicating a plurality of processes;
generating a resource requirement indicating requested physical nodes of a plurality of physical nodes available to execute processes, and an assignment of the processes to execute on the requested physical nodes;
determining physical nodes having sufficient resources for the processes assigned to the requested physical node;
returning a resource allocation of the determined physical nodes having the sufficient resources for the processes assigned to the requested physical nodes; and
updating the execution plan to generate an updated execution plan indicating the determined physical nodes for the requested physical nodes on which the processes assigned to the requested physical nodes will execute according to the resource allocation.

10. The system of claim 9 , wherein the determining the physical nodes performs for each requested physical node of the physical nodes comprises:
determining whether there is one physical node having sufficient resources for at least one process allocated to the requested physical node;
allocating the determined one physical node to the at least one process assigned to the requested physical node in response to determining that there is one physical node having the sufficient resources; and
allocating multiple physical nodes for the at least one process allocated to the requested physical node when there is not one physical node having sufficient resources for the processes assigned to the requested physical node.

11. The system of claim 10 , wherein the processes include section leaders and players, wherein each of the players are assigned to one of a plurality of section leaders, wherein the resource requirement assigns each of the section leaders and the players assigned to the section leader to one of the requested physical nodes to prefer an allocation of each of the section leaders and the players assigned to the section leader to a separate of the physical nodes.

12. The system of claim 10 , wherein the allocating multiple physical nodes for one of the requested physical nodes comprises allocating a minimum number of physical nodes having sufficient resources for the processes that are assigned to the requested physical node.

13. The system of claim 10 , wherein the execution plan is for an extract, transform, and load (“ETL”) program, wherein the processes include a conductor, section leaders, and players assigned to the section leaders, wherein the conductor manages execution of the section leaders and the players, and wherein the section leaders manage execution of players assigned to the section leaders.

14. The system of claim 9 , wherein the operations further comprise:
generating a configuration file template providing placeholder names for the requested physical nodes, wherein the generated execution plan assigns the processes to execute on the placeholder names for the requested physical nodes.

15. A method for obtaining resources for an execution plan, comprising:
generating an execution plan indicating a plurality of processes;
generating a resource requirement indicating requested physical nodes of a plurality of physical nodes available to execute processes, and an assignment of the processes to execute on the requested physical nodes;
determining physical nodes having sufficient resources for the processes assigned to the requested physical nodes;
returning a resource allocation of the determined physical nodes having the sufficient resources for the processes assigned to the requested physical nodes; and
updating t

Assignees

• IBM

Inventors

• Bonagiri Krishna K
• Jacobson Eric A
• Li Yong
• Pu Xiaoyan

Classifications

• G06F9/48

  Program initiating; Program switching, e.g. by interrupt · CPC title

• G06F9/44

  Arrangements for executing specific programs · CPC title

• G06F9/5066

  Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs (mappping at compile time, see G06F8/451) · CPC title

• H04L41/5054 Primary

  Automatic deployment of services triggered by the service manager, e.g. service implementation by automatic configuration of network components · CPC title

• H04L41/0843

  based on generic templates · CPC title