Multi-directional reduction in large scale deep-learning

Patent metadata

Field	Value
Publication number	US-10922606-B2
Application number	US-201715621258-A
Country	US
Kind code	B2
Filing date	Jun 13, 2017
Priority date	Jun 13, 2017
Publication date	Feb 16, 2021
Grant date	Feb 16, 2021

Abstract

A method for executing multi-directional reduction algorithms includes identifying a set of nodes, wherein a node includes at least one data element, creating a set of partitions including one or more data elements from at least two nodes, wherein the at least two nodes are arranged in a single direction with respect to the positioning of the set of nodes, executing a reduction algorithm on the data elements within the created set of partitions, creating an additional set of partitions including one or more data elements from at least two nodes, wherein the at least two nodes are arranged in a different direction with respect to the positioning of the set of nodes, executing a reduction algorithm on the data elements within the created additional set of partitions, and providing a set of reduced results corresponding to the at least one data element.

First claim

What is claimed is:

1. A computer implemented method for executing multi-directional reduction algorithms, the method comprising:
identifying a set of one or more nodes, wherein a node of the set of one or more nodes includes at least one data element;
creating a set of partitions including one or more data elements from at least two nodes of the set of nodes, wherein the at least two nodes of the set of nodes are arranged in a single direction with respect to the positioning of the set of one or more nodes;
executing a reduction algorithm on the data elements within the created set of partitions, wherein the data elements from within a shared partition are reduced together;
creating an additional set of partitions including one or more data elements from at least two nodes of the set of nodes, wherein the at least two nodes of the set of nodes are arranged in a different direction with respect to the positioning of the set of one or more nodes;
executing a reduction algorithm on the data elements within the created additional set of partitions, wherein the data elements from within a shared partition are reduced together; and
providing a set of reduced results corresponding to the at least one data element.

2. The method of claim 1 , further comprising creating a logical arrangement of the identified set of one or more nodes.

3. The method of claim 2 , wherein the set of nodes is arranged in an n-dimensional space, and wherein the created logical arrangement of the identified set of one or more nodes is arranged in an n-dimensional array.

4. The method of claim 1 , further comprising distributing a set of reduced results to each node in the identified set of one or more nodes.

5. The method of claim 1 , wherein multiple loop-based reduction algorithms are executed simultaneously on multiple partitions.

6. A computer program product comprising:
one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising instructions to:
identify a set of one or more nodes, wherein a node of the set of one or more nodes includes at least one data element;
create a set of partitions including one or more data elements from at least two nodes of the set of nodes, wherein the at least two nodes of the set of nodes are arranged in a single direction with respect to the positioning of the nodes;
execute a reduction algorithm on the data elements within the created set of partitions, wherein the data elements from within a shared partition are reduced together;
create an additional set of partitions including one or more data elements from at least two nodes of the set of nodes, wherein the at least two nodes of the set of nodes are arranged in a different direction with respect to the positioning of the nodes;
execute a reduction algorithm on the data elements within the created additional set of partitions, wherein the data elements from within a shared partition are reduced together; and
provide a set of reduced results corresponding to the at least one data element.

7. The computer program product of claim 6 , further comprising instructions to create a logical arrangement of the identified set of one or more nodes.

8. The computer program product of claim 7 , wherein the set of nodes is arranged in an n-dimensional space, and wherein the created logical arrangement of the identified set of one or more nodes is arranged in an n-dimensional array.

9. The computer program product of claim 6 , further comprising instructions to distribute a set of reduced results to each node in the identified set of one or more nodes.

10. The computer program product of claim 6 , wherein multiple loop-based reduction algorithms are executed simultaneously on multiple partitions.

11. A computer system for generating natural language processing queries, the computer system comprising:
one or more computer processors;
one or more computer-readable storage media;
program instructions stored on the computer-readable storage media for execution by at least one of the one or more processors, the program instructions comprising instructions to:
identify a set of one or more nodes, wherein a node of the set of one or more nodes includes at least one data element;
create a set of partitions including one or more data elements from at least two nodes of the set of nodes, wherein the at least two nodes of the set of nodes are arranged in a single direction with respect to the positioning of the nodes;
execute a reduction algorithm on the data elements within the created set of partitions, wherein the data elements from within a shared partition are reduced together;
create an additional set of partitions including one or more data elements from at least two nodes of the set of nodes, wherein the at least two nodes of the set of nodes are arranged in a different direction with respect to the positioning of the nodes;
execute a reduction algorithm on the data elements within the created additional set of partitions, wherein the data elements from within a shared partition are reduced together; and
provide a set of reduced results corresponding to the at least one data element.

12. The computer system of claim 11 , further comprising instructions to create a logical arrangement of the identified set of one or more nodes.

13. The computer system of claim 12 , wherein the set of nodes is arranged in an n-dimensional space, and wherein the created logical arrangement of the identified set of one or more nodes is arranged in an n-dimensional array.

14. The computer system of claim 11 , further comprising instructions to distribute a set of reduced results to each node in the identified set of one or more nodes.

15. The computer system of claim 11 , wherein multiple loop-based reduction algorithms are executed simultaneously on multiple partitions.

16. A computer implemented method comprising:
identifying a set of one or more nodes arranged in a T-dimensional space, wherein a node of the set of one or more nodes includes at least one data element;
dividing the set of nodes into T partitions of nodes, wherein a partition includes one or more data elements from at least two nodes of the set of nodes;
executing a loop-based reduction algorithm in a selected direction on the data elements within the T partitions of nodes, wherein the data elements from within a shared partition are reduced together;
shifting the T partitions by one direction to provide a next direction for reduction;
executing an additional loop-based reduction algorithm in the next direction on the data elements within the T partitions of nodes; and
providing a set of reduced results corresponding to the at least one data element.

17. The method of claim 16 , further comprising creating a logical arrangement of the identified set of one or more nodes.

18. The method of claim 17 , wherein the created logical arrangement of the identified set of one or more nodes is arranged in a T-dimensional array.

19. The method of claim 16 , further comprising distributing a set of reduced results to each node in the identified set of one or more nodes.

20. The method of claim 16 , wherein multiple loop-based reduction algorithms are executed simultaneously on multiple partitions.

21. A computer program product comprising:
one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions

Assignees

• IBM

Inventors

• Cho Minsik
• Finkler Ulrich A
• Kung David S
• Zhang Li

Classifications

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/063 Primary

  using electronic means · CPC title

• G06N5/04

  Inference or reasoning models · CPC title