Data processing apparatus and method for handling stalled data

Patent metadata

Field	Value
Publication number	US-12481606-B2
Application number	US-202218574277-A
Country	US
Kind code	B2
Filing date	Jun 21, 2022
Priority date	Jul 1, 2021
Publication date	Nov 25, 2025
Grant date	Nov 25, 2025

Abstract

There is provided a data processing apparatus and method. The data processing apparatus comprises a plurality of processing elements connected via a network arranged on a single chip to form a spatial architecture. Each processing element comprising processing circuitry to perform processing operations and memory control circuitry to perform data transfer operations and to issue data transfer requests for requested data to the network. The memory control circuitry is configured to monitor the network to retrieve the requested data from the network. Each processing element is further provided with local storage circuitry comprising a plurality of local storage sectors to store data associated with the processing operations, and auxiliary memory control circuitry to monitor the network to detect stalled data (S 60 ). The auxiliary memory control circuitry is configured to transfer the stalled data from the network to an auxiliary storage buffer (S 66 ) dynamically selected from amongst the plurality of local storage sectors (S 64 ).

First claim

The invention claimed is:

1 . A data processing apparatus comprising a plurality of processing elements connected via a network arranged on a single chip to form a spatial architecture, each processing element of the plurality of processing elements comprising:
processing circuitry to perform processing operations; memory control circuitry to perform data transfer operations associated with the processing element and to issue data transfer requests for requested data to the network, wherein the memory control circuitry is configured to monitor the network for the requested data and, in response to detecting the requested data, to retrieve the requested data from the network; local storage circuitry to store data associated with the processing operations, the local storage circuitry comprising a plurality of local storage sectors; and auxiliary memory control circuitry configured to monitor the network to detect stalled data associated with the processing element and, in response to detecting the stalled data, to transfer the stalled data from the network to an auxiliary storage buffer dynamically selected from amongst the plurality of local storage sectors.

2 . The data processing apparatus of claim 1 , wherein:
the processing operations and the data transfer operations are triggered operations; each triggered operation is performed in response to trigger data meeting a corresponding processing trigger condition, and the processing element is configured to set further trigger data in response to completion of the triggered operation.

3 . The data processing apparatus of claim 1 , wherein:
each data transfer request specifies a data request tag; and the auxiliary memory control circuitry comprises auxiliary table storage to store an auxiliary memory table associating a corresponding data request tag of the stalled data and a corresponding location in the auxiliary storage buffer.

4 . The data processing apparatus of claim 3 , wherein the auxiliary memory control circuitry is configured to perform, in response to an indication that the memory control circuitry requires the stalled data, a lookup in the auxiliary memory table based on the data request tag specified by the data transfer request.

5 . The data processing apparatus of claim 4 , wherein:
the auxiliary memory control circuitry is configured to, when the lookup hits in the auxiliary memory table, provide a location associated with the data request tag to the memory control circuitry; and the memory control circuitry is configured to retrieve the stalled data from the location.

6 . The data processing apparatus of claim 1 , wherein auxiliary memory control circuitry is configured to select the auxiliary storage buffer dynamically from amongst the plurality of local storage sectors based on a usage metric of each of the local storage sectors by the memory control circuitry.

7 . The data processing apparatus of claim 6 , wherein:
each processing element further comprises a plurality of counters defining the usage metric, each counter indicative of a number of times an associated local storage sector has been accessed by the memory control circuitry; and the auxiliary memory control circuitry is configured to select, as the auxiliary storage buffer, a particular local storage sector corresponding to a counter of the plurality of counters indicating fewest accesses.

8 . The data processing apparatus of claim 7 , wherein the local storage is configured to allow the memory control circuitry and the auxiliary memory control circuitry to access different sectors in parallel.

9 . The data processing apparatus of claim 8 , wherein the local storage circuitry is configured to, in response to parallel accesses to a same sector by the memory control circuitry and the auxiliary control circuitry, prioritize the auxiliary memory control circuitry.

10 . The data processing apparatus of claim 1 , wherein:
the memory control circuitry is configured to, when retrieving the data from the network, modify a dequeue signal to indicate that the data has been removed from the network; and the auxiliary memory control circuitry is configured to, when monitoring the network to detect the stalled data: periodically monitor the dequeue signal; and determine that the queued data comprises stalled data when the dequeue signal indicates that the data remains on the network.

11 . The data processing apparatus of claim 1 , wherein:
each processing element further comprises a plurality of interface channels to store queued data to be transferred between the processing element and the network; and the memory control circuitry is configured to monitor the network for the requested data by monitoring the plurality of interface channels.

12 . The data processing apparatus of claim 11 , wherein the auxiliary memory control circuitry is configured to, when monitoring the network to detect stalled data:
capture first queue data indicative of the queued data in the plurality of interface channels at the start of a predetermined time period; capture second queue data indicative of the queued data in the plurality of interface channels at the end of the predetermined time period; and determine that the queued data comprises stalled data when the first queue data is the same as the second queue data.

13 . The data processing apparatus of claim 1 , wherein the spatial architecture comprises the plurality of processing elements connected to form a multi-dimensional array.

14 . The data processing apparatus of claim 13 , wherein:
the data processing apparatus is configured to arrange processing elements of the plurality of processing elements into subgroups; and the data processing apparatus further comprises a multiplexer configured to route memory requests to the subgroup containing the processing element.

15 . The data processing apparatus of claim 1 , wherein the data transfer request is one of:
an inter-processing element data transfer request specifying data to be transferred from another processing element; and a memory request specifying data to be transferred from a memory location in main memory.

16 . The data processing apparatus of claim 1 , wherein the plurality of processing elements are connected to global storage via a common interface node.

17 . The data processing apparatus of claim 1 , wherein the data transfer request specifies that the returned data is static data to be stored in a particular storage location.

18 . The data processing apparatus of claim 17 , wherein the auxiliary memory control circuitry is configured to, in response to detecting that the stalled data is static data, transfer the stalled data from the network to the particular storage location.

19 . A non-transitory computer-readable medium storing computer-readable code for fabrication of the data processing apparatus of claim 1 .

20 . A method of operating a data processing apparatus comprising a plurality of processing elements connected via a network arranged on a single chip to form a spatial architecture, each processing element comprising processing circuitry, memory control circuitry, local storage circuitry comprising a plurality of local storage sectors and auxiliary memory control circuitry, the method comprising:
performing processing operations using the processing circuitry; storing data associated with the processing operations; with the memory control circuitry, performing data transfer operations associated with the processing

Assignees

• Advanced Risc Mach Ltd

Inventors

• Eyole Mbou
• Gabrielli Giacomo
• VENU Balaji

Classifications

• G06F13/26

  with priority control · CPC title

• G06F13/161

  with latency improvement · CPC title

• G06F2213/0038

  System on Chip · CPC title

• G11C7/1006

  Data managing, e.g. manipulating data before writing or reading out, data bus switches or control circuits therefor · CPC title

• G06F12/0284

  Multiple user address space allocation, e.g. using different base addresses (interprocessor communication G06F15/163) · CPC title