Data shuffle offload

Patent metadata

Field	Value
Publication number	US-11934332-B2
Application number	US-202217590339-A
Country	US
Kind code	B2
Filing date	Feb 1, 2022
Priority date	Feb 1, 2022
Publication date	Mar 19, 2024
Grant date	Mar 19, 2024

Abstract

Devices, methods, and systems are provided. In one example, a device is described to include a device interface that receives data from at least one data source; a data shuffle unit that collects the data received from the at least one data source, receives a descriptor that describes a data shuffle operation to perform on the data received from the at least one data source, performs the data shuffle operation on the collected data to produce shuffled data, and provides the shuffled data to at least one data target.

First claim

What is claimed is:

1. A network device, comprising:
a device interface that receives data from at least one data source; and
a data shuffle unit that collects the data received from the at least one data source, receives a descriptor that describes a data shuffle operation to perform on the data received from the at least one data source, performs the data shuffle operation on the collected data to produce shuffled data, and provides the shuffled data to at least one data target, wherein the descriptor comprises at least one of a work queue element (WQE) posted to a queue pair, a memory region description, a description of a Remote Direct Memory Access (RDMA) request, and a description of an application-level request.

2. The network device of claim 1 , wherein the data shuffle operation is done according to a descriptor provided by the at least one data source.

3. The network device of claim 1 , wherein the data shuffle unit comprises a processor and memory, wherein the memory is used to store the data received from the at least one data source until a predetermined amount of data is collected, and wherein the processor performs the shuffle operation on the predetermined amount of data stored in memory.

4. The network device of claim 1 , wherein the at least one data source comprises a host memory device.

5. The network device of claim 1 , wherein the at least one data source comprises an on-network device memory.

6. The network device of claim 1 , wherein the at least one data source comprises a peer memory device.

7. The network device of claim 1 , wherein the data is received in a plurality of network packets.

8. The network device of claim 1 , wherein the at least one data target comprises a plurality of data targets.

9. The network device of claim 1 , wherein the at least one data target comprises at least one of a host memory device, a peer memory device, and an on-network device memory.

10. The network device of claim 1 , wherein the at least one data target is located remotely from the data shuffle unit.

11. The network device of claim 10 , further comprising:
a second device interface that couples the network device with the data target, wherein the device interface comprises a communication port and wherein the second device interface also comprises a communication port.

12. The network device of claim 1 , wherein the at least one data source is located remotely from the data shuffle unit.

13. The network device of claim 1 , wherein the descriptor comprises the WQE posted to the queue pair and represents a single data shuffle operation.

14. The network device of claim 1 , wherein the descriptor is obtained from a memory device of the shuffle unit and comprises a memory region description that is usable to perform multiple shuffle operations.

15. The network device of claim 1 , wherein the descriptor is received in a network packet via the device interface.

16. The network device of claim 1 , wherein the descriptor is received as part of the RDMA request or the application-level request.

17. The network device of claim 1 , wherein the device interface and the shuffle unit are provided as part of a Network Interface Controller (NIC).

18. The network device of claim 1 , wherein the device interface and the shuffle unit are provided as part of a network switch.

19. The network device of claim 1 , wherein the data shuffle operation comprises at least one of a matrix transpose, a non-homogenous transpose, a removal of padding bits, an addition of padding bits, a tensor layout conversion, a bit packing, a component packing, and a bit tiling.

20. The network device of claim 1 , wherein an application receiving the shuffled data is unaware of the data shuffle performed by the data shuffle unit.

21. The network device of claim 1 , wherein the shuffled data is provided to the at least one data target via the device interface.

22. The network device of claim 1 , wherein the shuffled data comprises fewer bits than the collected data.

23. The network device of claim 1 , wherein the at least one data source comprises a network edge device.

24. The network device of claim 1 , wherein the device interface comprises a serial data interface and wherein the data is received from the at least one data source via a serial communication protocol.

25. A system, comprising:
a device interface that receives data from at least one data source;
circuitry; and
memory coupled with the circuitry, wherein instructions stored in memory enable the circuitry to collect the data received from the at least one data source, receive a descriptor that describes a data shuffle operation to perform on the data received from the at least one data source, perform the data shuffle operation on the collected data to produce shuffled data, and then provide the shuffled data to the at least one data target, wherein the descriptor comprises at least one of a work queue element (WQE) posted to a queue pair, a memory region description, a description of a Remote Direct Memory Access (RDMA) request, and a description of an application-level request.

Assignees

• Mellanox Technologies Ltd

Inventors

• Marcovitch Daniel
• Levi Dotan David
• Srebro Eyal
• Peretz Eliel
• Moyal Roee
• Graham Richard
• Bloch Gil
• Pieper Sean

Classifications

• G06F13/42 Primary

  Bus transfer protocol, e.g. handshake; Synchronisation · CPC title

• G06F7/76

  Arrangements for rearranging, permuting or selecting data according to predetermined rules, independently of the content of the data · CPC title

• G06F9/3004 Primary

  to perform operations on memory · CPC title

• G06F9/5027

  the resource being a machine, e.g. CPUs, Servers, Terminals · CPC title