Allocating physical nodes for processes in an execution plan

Patent metadata

Field	Value
Publication number	US-9787761-B2
Application number	US-201414500896-A
Country	US
Kind code	B2
Filing date	Sep 29, 2014
Priority date	Sep 29, 2014
Publication date	Oct 10, 2017
Grant date	Oct 10, 2017

Abstract

Provided are a computer program product, system, and method for allocating physical nodes for processes in an execution plan. An execution plan is generated indicating a plurality of processes. A resource requirement is generated indicating requested physical nodes and an assignment of the processes to execute on the requested physical nodes. A determination is made from the resource requirement of a resource allocation of physical nodes for the requested physical nodes and the processes. The execution plan is updated to generate an updated execution plan indicating the physical nodes on which the processes will execute according to the received resource allocation.

First claim

What is claimed is:

1. A computer program product for obtaining resources for an execution plan, comprising a computer readable storage medium having computer readable program code embodied therein that when executed performs operations, the operations comprising:
generating an execution plan indicating a plurality of processes;
generating a resource requirement indicating a plurality of requested physical nodes and an assignment of the processes to execute on the requested physical nodes;
determining from the resource requirement a resource allocation of physical nodes for the requested physical nodes and the processes by:
determining whether there is one physical node having sufficient resources for the processes allocated to a requested physical node of the requested physical nodes;
allocating the determined one physical node to the processes assigned to the requested physical node in response to determining that there is one physical node having the sufficient resources; and
in response to determining that there is no one physical node having sufficient resources for the processes assigned to the requested physical node, determining whether there is a soft constraint or hard constraint indicated for the processes assigned to the requested physical node;
allocating multiple physical nodes for the processes allocated to the requested physical node in response to determining that the soft constraint is indicated for at least one of the processes assigned to the requested physical node for which no one physical node is determined when there is not one physical node having sufficient resources for the processes assigned to the requested physical node; and
updating the execution plan to generate an updated execution plan indicating the physical nodes on which the processes will execute according to the resource allocation.

2. The computer program product of claim 1 , wherein the allocating multiple physical nodes for one of the requested physical nodes comprises allocating a minimum number of physical nodes having sufficient resources for the processes that are assigned to the requested physical node.

3. The computer program product of claim 1 , wherein the operations further comprise:
returning fail to a request to allocate physical nodes in response to determining that the hard constraint is indicated for the processes.

4. The computer program product of claim 1 , wherein the processes include section leaders and players, wherein each of the players are assigned to one of a plurality of section leaders, wherein the resource requirement assigns each of the section leaders and the players assigned to the section leader to one of the requested physical nodes to prefer an allocation of each of the section leaders and the players assigned to the section leader to a separate of the physical nodes.

5. The computer program product of claim 1 , wherein the operations further comprise:
generating a configuration file template providing placeholder names for the requested physical nodes, wherein the generated execution plan assigns the processes to execute on the placeholder names for the requested physical nodes.

6. The computer program product of claim 1 , wherein the execution plan is for an extract, transform, and load (“ETL”) program, wherein the processes include a conductor, section leaders, and players assigned to the section leaders, wherein the conductor manages execution of the section leaders and the players, and wherein the section leaders manage execution of players assigned to the section leaders.

7. The computer program product of claim 1 , wherein the physical nodes assigned differ in quantity from a number of the requested physical nodes in the resource requirement.

8. The computer program product of claim 1 , wherein the allocated physical nodes provide multiple physical nodes for one of the requested physical nodes, and wherein the processes assigned to the requested physical node for which physical nodes were allocated, execute on the multiple physical nodes.

9. A computer program product for obtaining resources for an execution plan, comprising a computer readable storage medium having computer readable program code embodied therein that when executed performs operations, the operations comprising:
generating an execution plan indicating a plurality of section leaders and players, wherein each of the players are assigned to a second leader of the plurality of section leaders;
generating a resource requirement indicating a plurality of requested physical nodes and an assignment of processes to execute on the requested physical nodes, wherein the resource requirement indicates, for each of the players, one of a soft constraint and a hard constraint, wherein the soft constraint for the player indicates a preference to allocate the player to a physical node of the physical nodes on which the section leader for the player is allocated, wherein the preference of the soft constraint can be relaxed if one physical node cannot be located having sufficient resources to execute all the players of the section leader, wherein the hard constraint for the player indicates a requirement to assign the player to the physical node including the section leader for the player;
determining from the resource requirement a resource allocation of physical nodes for the requested physical nodes and the section leaders and players by:
determining whether there is one physical node having sufficient resources for the section leader and players allocated to the requested physical node;
allocating the determined one physical node to the section leader and players assigned to the requested physical node in response to determining that there is one physical node having the sufficient resources; and
allocating multiple physical nodes for the section leaders and players allocated to the requested physical node when there is not one physical node having sufficient resources for the section leader and players assigned to the requested physical node; and
updating the execution plan to generate an updated execution plan indicating the physical nodes on which the section leaders and players will execute according to the resource allocation.

10. The computer program product of claim 9 , wherein the operations further comprise:
in response to determining that there is no one physical node having sufficient resources for the players assigned to the section leader for one of the requested physical nodes, determining whether all players assigned to the section leader having the hard constraint can be allocated to one physical node having the section leader; and
in response to determining that all players having the hard constraint can be allocated to the one physical node having the section leader, performing:
allocating the section leader and the players having the hard constraint to a first physical node having sufficient resources; and
allocating at least one of the players having the soft constraint to a second physical node.

11. The computer program product of claim 10 , wherein the operations further comprise:
returning fail to a request to allocate physical nodes in response to determining that there is not one physical node having sufficient resources for the section leader and all the players assigned to the section leader having the hard constraint and the soft constraint.

12. The computer program product of claim 9 , wherein the resource allocation allocates a player of the players having the soft constraint to execute on one of the physical nodes that is not also allocated to the section leader of the player having the soft constraint.

13. A system for obtai

Assignees

• IBM

Inventors

• Bonagiri Krishna K
• Jacobson Eric A
• Li Yong
• Pu Xiaoyan

Classifications

• G06F9/5066

  Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs (mappping at compile time, see G06F8/451) · CPC title

• H04L67/10 Primary

  in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) · CPC title