Data shuffle offload

Patent metadata

Field	Value
Publication number	US-12229072-B2
Application number	US-202418598382-A
Country	US
Kind code	B2
Filing date	Mar 7, 2024
Priority date	Feb 1, 2022
Publication date	Feb 18, 2025
Grant date	Feb 18, 2025

Abstract

Devices, methods, and systems are provided. In one example, a device is described to include a device interface that receives data from at least one data source; a data shuffle unit that collects the data received from the at least one data source, receives a descriptor that describes a data shuffle operation to perform on the data received from the at least one data source, performs the data shuffle operation on the collected data to produce shuffled data, and provides the shuffled data to at least one data target.

First claim

What is claimed is:

1. A system, comprising:
a device interface to receive data from at least one data source; and
circuitry to collect the data received from the at least one data source, receive a descriptor that describes a data shuffle operation to perform on the data received from the at least one data source, perform the data shuffle operation on the data to produce shuffled data, and then provide the shuffled data to at least one data target, wherein the descriptor comprises at least one of a processor instruction description, a work queue element (WQE) posted to a queue pair, a memory region description, a description of a Remote Direct Memory Access (RDMA) request, and a description of an application-level request.

2. The system of claim 1 , wherein the data shuffle operation is performed according to a descriptor provided by the at least one data source.

3. The system of claim 1 , wherein the data received from the at least one data source is stored until a predetermined amount of data is collected.

4. The system of claim 1 , wherein the at least one data source comprises a host memory device.

5. The system of claim 1 , wherein the at least one data source comprises an on-network device memory.

6. The system of claim 1 , wherein the at least one data source comprises a peer memory device.

7. The system of claim 1 , wherein the data is received in a plurality of network packets.

8. The system of claim 1 , wherein the at least one data target comprises a plurality of data targets.

9. The system of claim 1 , wherein the at least one data target comprises at least one of a host memory device, a peer memory device, and an on-network device memory.

10. The system of claim 1 , wherein the at least one data target is located remotely from the circuitry and further comprising:
a second device interface that communicates with the at least one data target, wherein the device interface comprises a communication port and wherein the second device interface also comprises a communication port.

11. The system of claim 1 , wherein the at least one data source is located remotely from the circuitry.

12. The system of claim 1 , wherein the descriptor comprises the WQE posted to the queue pair and represents a single data shuffle operation.

13. The system of claim 1 , wherein the descriptor is obtained from a memory device and comprises a memory region description that is usable to perform multiple shuffle operations.

14. The system of claim 1 , wherein the descriptor is received in a network packet via the device interface.

15. The system of claim 1 , wherein the descriptor is received as part of the RDMA request or the application-level request.

16. The system of claim 1 , wherein the device interface and the circuitry are provided as part of a Network Interface Controller (NIC).

17. The system of claim 1 , wherein the device interface and the circuitry are provided as part of a network switch.

18. The system of claim 1 , wherein the data shuffle operation comprises at least one of a matrix transpose, a non-homogenous transpose, a removal of padding bits, an addition of padding bits, a tensor layout conversion, a bit packing, a component packing, and a bit tiling.

19. The system of claim 1 , wherein an application receiving the shuffled data is unaware of the data shuffle operation performed by the circuitry.

20. A system, comprising:
circuitry in communication with a device interface that receives data from at least one data source, wherein the circuitry collects the data received from the device interface, receives a descriptor that describes a data shuffle operation to perform on the data received from the at least one data source, performs the data shuffle operation on the data to produce shuffled data, and then provides the shuffled data to at least one data target, wherein the descriptor comprises at least one of a processor instruction description, a work queue element (WQE) posted to a queue pair, a memory region description, a description of a Remote Direct Memory Access (RDMA) request, and a description of an application-level request.

Assignees

• Mellanox Technologies Ltd

Inventors

• Marcovitch Daniel
• Levi Dotan David
• Srebro Eyal
• Peretz Eliel
• Moyal Roee
• Graham Richard
• Bloch Gil
• Pieper Sean

Classifications

• G06F7/76

  Arrangements for rearranging, permuting or selecting data according to predetermined rules, independently of the content of the data · CPC title

• G06F9/5027

  the resource being a machine, e.g. CPUs, Servers, Terminals · CPC title

• G06F13/42 Primary

  Bus transfer protocol, e.g. handshake; Synchronisation · CPC title

• G06F9/3004 Primary

  to perform operations on memory · CPC title