Method and apparatus for selecting data access method in a heterogeneous processing system with multiple processors

Patent metadata

Field	Value
Publication number	US-12229057-B2
Application number	US-202318099032-A
Country	US
Kind code	B2
Filing date	Jan 19, 2023
Priority date	Jan 19, 2023
Publication date	Feb 18, 2025
Grant date	Feb 18, 2025

Abstract

A heterogeneous processing system including a host processor coupled to a host memory, a first processor coupled to a first memory, a second processor coupled to a second memory, multiple data transfer resources, and switch and bus circuitry that communicatively couples the host processor, the first processor, the second processor, and the data transfer resources. The host processor is configured to detect an application for execution by both the first processor and the second processor, to select one of multiple data transfer methods for transferring data between the first and second processors, and to configure the heterogeneous processing system based on the selected data transfer method. The data transfer methods include memory extension operation, one memory to memory transfer operation, and two memory to memory transfer operation using at least one intermediate host buffer.

First claim

What is claimed is:

1. A heterogeneous processing system, comprising:
switch and bus circuitry;
a host processor coupled to a host memory accessible from the host processor without utilizing the switch and bus circuitry;
a first processor coupled to a first memory accessible from the first processor without utilizing the switch and bus circuitry;
a second processor coupled to a second memory accessible from the second processor without utilizing the switch and bus circuitry; and
a plurality of data transfer resources;
wherein the switch and bus circuitry communicatively couples the host processor, the first processor, the second processor, and the plurality of data transfer resources;
wherein the host processor is configured to, at runtime:
i) detect an application for execution by both the first processor and the second processor,
ii) dynamically select one of a plurality of different data access methods for accessing data between the first and second processors based on user-defined metadata of data passing, including:
source and destination device types,
memory addresses,
bandwidth of application-defined pipeline, and
latency requirement of operation device stages, and

iii) configure the heterogeneous processing system based on the selected data access method;

wherein the plurality of different data access methods for accessing data between the first and second processors include:
a) directly accessing the data in the second memory by the first processor,
b) transferring the data between the first memory and the second memory using a first data transfer resource of the plurality of data transfer resources without passing the data through the host memory, and
c) transferring the data from the second memory to the host memory and then transferring the data from the host memory to the first memory using one or more data transfer resources of the plurality of data transfer resources.

2. The heterogeneous processing system of claim 1 , wherein the host processor selects memory extension operation by mapping virtual addresses of the second memory to physical addresses of the switch and bus circuitry and by configuring the first processor to directly access the second memory using the mapped physical addresses according to the memory extension operation.

3. The heterogeneous processing system of claim 2 , wherein the second processor is programmed to execute a first part of the application to generate and write first data into the second memory, and wherein the first processor is configured to directly access the first data from the second memory while executing a second part of the application using the first data.

4. The heterogeneous processing system of claim 3 , wherein the first processor is configured to directly access the host memory and to write second data output from executing the second part of the application directly into the host memory.

5. The heterogeneous processing system of claim 2 , wherein the first processor is configured to execute a first part of the application to generate first data and to directly write the first data into the second memory.

6. The heterogeneous processing system of claim 2 , wherein the first processor is configured to directly read first data from the host memory while executing a first part of the application using the first data to generate second data and to directly write the second data into the second memory while executing the first part of the application.

7. The heterogeneous processing system of claim 1 , wherein the host processor is configured to map virtual addresses of the second memory to physical addresses of the switch and bus circuitry and to configure a first data transfer resource to perform a one memory to memory transfer operation between the first memory and the second memory.

8. The heterogeneous processing system of claim 7 , wherein the host processor is configured to program the first data transfer resource to directly transfer data between the first memory and the second memory using the mapped physical addresses.

9. The heterogeneous processing system of claim 7 , wherein the second processor is programmed to execute a first part of the application to generate and write first data into the second memory, and wherein the host processor is configured to prompt the first data transfer resource to transfer the first data from the second memory to the first memory.

10. The heterogeneous processing system of claim 9 , wherein the first processor is configured to execute a second part of the application using the first data and write second data into the first memory, and wherein the host processor is configured to prompt the first data transfer resource to transfer the second data into the host memory.

11. The heterogeneous processing system of claim 7 , wherein the first processor is configured to execute a first part of the application to generate and write first data into the first memory, and wherein the host processor is configured to prompt the first data transfer resource to transfer the first data from the first memory to the second memory.

12. The heterogeneous processing system of claim 11 , wherein the second processor is programmed to execute a second part of the application using the first data and write second data into the second memory.

13. The heterogeneous processing system of claim 12 , wherein the host processor is configured to prompt the second data transfer resource to transfer the second data from the second memory into the host memory.

14. The heterogeneous processing system of claim 1 , wherein the host processor selects a two memory to memory transfer operation by allocating buffer space within the host memory, wherein the first processor is configured to execute a first part of the application which generates and writes first data into the first memory, and wherein the host processor is further configured to program the plurality of data transfer resources to transfer the first data from the first memory to the buffer space of the host memory and to transfer the first data from the buffer space of the host memory into the second memory.

15. The heterogeneous processing system of claim 14 , wherein the buffer space comprises a common buffer that is allocated within a non-swappable portion of the host memory.

16. The heterogeneous processing system of claim 14 , wherein the second processor is configured to execute a second part of the application and to write second data in the second memory, and wherein the host processor is configured to prompt one of the plurality of data transfer resources to transfer the second data from the second memory to the host memory.

17. The heterogeneous processing system of claim 1 , wherein the host processor is configured to select a most efficient of one of the plurality of different data access methods based on current latency and bandwidth requirements.

18. A method of selecting data access method in a heterogeneous processing system, wherein the heterogeneous processing system includes switch and bus circuitry, a host processor, a first processor, a second processor, a host memory accessible from the host processor without utilizing the switch and bus circuitry, a first memory accessible from the first processor without utilizing the switch and bus circuitry, a second memory accessible from the second processor without utilizing the switch and bus circuitry, a plurality of data transfer resources, wherein the switch and bus circuitry communicatively couples the host processor, the first processor, the second processo

Assignees

• Sambanova Systems Inc

Inventors

• Goel Arnav
• Sanghvi Neal
• Bai Jiayu
• Zheng Qi
• Kumar Ravinder

Classifications

• G06F13/1668 Primary

  Details of memory controller · CPC title

• G06F2212/657

  Virtual address space management · CPC title

• G06F12/1036 Primary

  for multiple virtual address spaces, e.g. segmentation (G06F12/1045 takes precedence) · CPC title