Collective communication system and methods

Patent metadata

Field	Value
Publication number	US-11876642-B2
Application number	US-202117495824-A
Country	US
Kind code	B2
Filing date	Oct 7, 2021
Priority date	Feb 25, 2019
Publication date	Jan 16, 2024
Grant date	Jan 16, 2024

Abstract

A method in which a plurality of process are configured to hold a block of data destined for other processes, with data repacking circuitry including receiving circuitry configured to receive at least one block of data from a source process of the plurality of processes, the repacking circuitry configured to repack received data in accordance with at least one destination process of the plurality of processes, and sending circuitry configured to send the repacked data to the at least one destination process of the plurality of processes, receiving a set of data for all-to-all data exchange, the set of data being configured as a matrix, the matrix being distributed among the plurality of processes, and transposing the data by each of the plurality of processes sending matrix data from the process to the repacking circuitry, and the repacking circuitry receiving, repacking, and sending the resulting matrix data to destination processes.

First claim

What is claimed is:

1. A method comprising:
providing a plurality of processes, each of said plurality of processes being configured to hold a block of data destined for others of said plurality of processes;
providing at least one instance of data repacking circuitry comprising: receiving circuitry configured to receive at least one block of data from at least one source process of the plurality of processes;
repacking circuitry configured to repack received data in accordance with at least one destination process of the plurality of processes; and
sending circuitry configured to send the repacked data to said at least one destination process of the plurality of processes;
receiving a set of data for all-to-all data exchange, the set of data being configured as a plurality of blocks of data in a matrix as matrix data, the matrix being distributed among the plurality of processes; and
transposing the matrix data by changing the position of selected blocks of data of the plurality of blocks of data relative to the other blocks of data of the plurality of the blocks of data, without changing the structure of each of the blocks of data, and sending the matrix data from said process to said data repacking circuitry; and said data repacking circuitry receiving, repacking, and sending the resulting matrix data to destination processes.

2. The method according to claim 1 and also comprising providing a control tree configured to control the plurality of processes and the repacking circuitry.

3. The method according to claim 2 and wherein the control tree comprises a reduction tree.

4. The method according to claim 3 , and also including providing assigning circuitry other than the control tree, the assigning circuitry being configured to assign a given source subgroup and a given destination subgroup to an instance of data repacking circuitry.

5. Apparatus comprising:
receiving circuitry configured to receive at least one block of data from at least one source process of a plurality of processes, each of said plurality of processes being configured to hold a block of data destined for others of said plurality of processes;
at least one instance of data repacking circuitry configured to repack received data in accordance with at least one destination process of the plurality of processes; and
sending circuitry configured to send the repacked data to said at least one destination process of the plurality of processes, the apparatus being configured to receive a set of data for all-to-all data exchange, the set of data being configured as a plurality of blocks of data in a matrix as matrix data, the matrix being distributed among the plurality of processes, and
the apparatus being further configured to transpose the matrix data by changing the position of selected blocks of data of the plurality of blocks of data relative to the other blocks of data of the plurality of the blocks of data, without changing the structure of each of the blocks of data, and sending the matrix data from said process at the repacking circuitry; and
the data repacking circuitry receiving, repacking, and sending the resulting matrix data to destination processes.

6. Apparatus according to claim 5 and also comprising a control tree configured to control the plurality of processes and the repacking circuitry.

7. Apparatus according to claim 6 and wherein the control tree comprises a reduction tree.

8. Apparatus according to claim 6 and also including assigning circuitry other than the control tree, the assigning circuitry being configured to assign a given source subgroup and a given destination subgroup to an instance of data repacking circuitry.

9. The method according to claim 2 , and wherein the control tree is further configured to:
receive registration messages from each of the plurality of processes;
mark a given subgroup of the plurality of processes as ready for operation when registration messages have been received from all members of said given subgroup;
when a given subgroup which is a source subgroup and a corresponding subgroup which is a destination subgroup are ready for operation, pair the given source subgroup and the given destination subgroup and assign the given source subgroup and the given destination subgroup to an instance of data repacking circuitry; and
notify each said source subgroup and each said destination subgroup when operations relating to each said source subgroup and each said destination subgroup have completed.

10. The method according to claim 9 , and wherein the control tree is configured, in addition to pairing the given source subgroup and the given destination subgroup, to assign the given source subgroup and the given destination subgroup to an instance of data repacking circuitry.

11. Apparatus according to claim 6 , and wherein the control tree is further configured to:
receive registration messages from each of the plurality of processes;
mark a given subgroup of the plurality of processes as ready for operation when registration messages have been received from all members of said given subgroup;
when a given subgroup which is a source subgroup and a corresponding subgroup which is a destination subgroup are ready for operation, pair the given source subgroup and the given destination subgroup and assign the given source subgroup and the given destination subgroup to an instance of data repacking circuitry; and
notify each said source subgroup and each said destination subgroup when operations relating to each said source subgroup and each said destination subgroup have completed.

12. Apparatus according to claim 11 , and wherein the control tree is configured, in addition to pairing the given source subgroup and the given destination subgroup, to assign the given source subgroup and the given destination subgroup to a given instance of data repacking circuitry.

13. A method comprising:
providing a plurality of processes, each of said plurality of processes being configured to hold a block of data destined for others of said plurality of processes;
providing at least one instance of data repacking circuitry comprising:
receiving circuitry configured to receive at least one block of data from at least one source process of the plurality of processes;
repacking circuitry configured to repack received data in accordance with at least one destination process of the plurality of processes; and
sending circuitry configured to send the repacked data to said at least one destination process of the plurality of processes;
receiving a set of data for all-to-all data exchange, the set of data being configured as a plurality of blocks of data in a matrix as matrix data, the matrix being distributed as one or more submatrices among the plurality of processes;
transposing the matrix data by changing the position of selected blocks of data of the plurality of blocks of data relative to the other blocks of data of the plurality of the blocks of data, without changing the structure of each of the blocks of data, and sending the matrix data from each submatrix from said each corresponding process to said data repacking circuitry;
said data repacking circuitry receiving, repacking, and sending the resulting matrix data to destination processes; and,
providing a control tree to assign each submatrix to an instance of data repackaging circuitry.

14. The method according to claim 13 , and wherein the control tree comprises a reduction tree.

15. The method according to claim 14 , and also including providing assigning circuitry other than the control tree, the assigning circuitry being configu

Assignees

• Mellanox Tech Tlv Ltd
• Mellanox Technologies Ltd

Inventors

• Graham Richard
• Levi Lion
• Bloch Gil
• Marcovitch Daniel
• Bloch Noam
• Qin Yong
• Blumenfeld Yaniv
• Zahavi Eitan

Classifications

• H04L12/40182 Primary

  by using a plurality of communication lines · CPC title

• G06F12/0246

  in block erasable memory, e.g. flash memory · CPC title

• H04B7/0456

  Selection of precoding matrices or codebooks, e.g. using matrices antenna weighting · CPC title

• H04L12/44

  Star or tree networks · CPC title

• H04W24/10

  Scheduling measurement reports {; Arrangements for measurement reports} · CPC title