Information processing apparatus and a method for processing a workload

Patent metadata

Field	Value
Publication number	US-11995475-B2
Application number	US-202017082864-A
Country	US
Kind code	B2
Filing date	Oct 28, 2020
Priority date	Oct 28, 2020
Publication date	May 28, 2024
Grant date	May 28, 2024

Abstract

An information processing apparatus is described for processing a workload. The information processing apparatus comprises a processor and a memory element connected to the processor via a data link. In advance of processing a workload, the information processing apparatus estimates an access time required to transfer an amount of the workload that is to be transferred from the external memory element to the processor, and estimates a processing time for the processor to process the workload. A processing rate characteristic of the processor and/or a data transfer rate between the memory and the processor is set in dependence upon the estimated processing time and estimated access time. Methods for varying a quality of service (QoS) value of requests to the external memory element are also described.

First claim

What is claimed is:

1. An information processing apparatus comprising:
a processor that has a processing rate characteristic;
a memory element connected to the processor via a data link, wherein the information processing apparatus is configured to control a data transfer rate between the memory element and the processor; and
a memory access engine connected to the data link configured to send requests to a memory controller;
wherein the information processing apparatus is configured to, in advance of processing a workload:
estimate an access time required to transfer an amount of the workload that is to be transferred between the external memory element and the processor;
estimate a processing time for the processor to process the workload; and
set at least one of the processing rate characteristic of the processor and the data transfer rate in dependence upon the estimated processing time and estimated access time, and
wherein the information processing apparatus is configured to:
calculate an amount of data that is to be transferred between the memory element and the processor at each of a plurality of stages of processing the workload;
determine a plurality of quality of service values to be included in requests to the memory controller based upon the calculated amount of data that is to be transferred at each stage;
send, by the memory access engine, requests to the memory controller including the determined quality of service values;
transfer data between the memory element and the processor in accordance with the data transfer rate; and
process, by the processor, the workload in accordance with the processing rate characteristic.

2. An information processing apparatus according to claim 1 , wherein the processor comprises one or more compute engines, and the processing rate characteristic is at least one of a compute engine clock rate and a number of compute engines operational within the processor.

3. An information processing apparatus according to claim 1 , wherein the workload is data relating to calculations for a layer of a neural network.

4. An apparatus according to claim 1 , wherein the processor comprises one or more compute engines, wherein the memory access engine, data link and memory element are located on a single power domain operating at a first voltage and the one or more compute engines are located in a second power domain configured to operate at a second voltage which is different from the first voltage.

5. An apparatus according to claim 4 , wherein the data transfer rate between the processor and the memory element is set by setting at least one of a frequency of the memory access engine and an operating frequency of the memory element.

6. An apparatus according to claim 1 , wherein the information processing apparatus is configured to estimate the processing time by determining an amount of computation required by the workload, determining a computational efficiency of the workload and using the amount of computation and the computational efficiency to estimate a processing time.

7. An apparatus according to claim 1 , wherein the information processing apparatus is configured to estimate the access time using information on data transfer rates from the memory element during one or more previous time windows.

8. An apparatus according to claim 1 , wherein the information processing apparatus is configured to set an operating frequency of the processor in dependence upon the estimated processing time and access time, and wherein a voltage supplied to the processor is set to be consistent with a maximum operating frequency of the processor set within a predetermined time period.

9. An apparatus according to claim 1 , wherein setting the data transfer rate comprises setting an operating frequency of the memory access engine in dependence upon the estimated processing time and access time, and wherein a voltage supplied to the memory access engine is set to be consistent with a maximum operating frequency of the set within a predetermined time period.

10. An apparatus according to claim 1 , wherein the memory access engine is configured to send up to a configurable number of outstanding requests to the memory controller.

11. An apparatus according to claim 1 , wherein the workload is data relating to calculations for a layer of a neural network and the information processing apparatus is configured to increase the quality of service to request a higher quality of service from the external memory element during processing of the start of the layer or immediately before the start of the layer.

12. An apparatus according to claim 1 , wherein the calculation of the amount of data that is to be transferred at each of the plurality of stages is performed by at least one of a processor driver and the processor.

13. An apparatus according to claim 1 , wherein the information processing apparatus is configured to monitor a rate at which data is received from the memory element and to adjust the determined quality of service values in dependence upon a monitored rate at which data is received from the memory element.

14. An apparatus according to claim 1 , wherein the processor is at least one of a Central Processing Unit, a Graphics Processing Unit, and a Neural Processing unit.

15. A method for processing a workload performed by an information processing apparatus comprising a processor that has a processing rate characteristic, a memory element connected to the processor via a data link, and a memory access engine connected to the data link configured to send requests to a memory controller, wherein the information processing apparatus is configured to control a data transfer rate between the memory element and the processor, and wherein the method comprises, in advance of processing a workload:
estimating an access time required to transfer an amount of the workload data that is to be transferred between the external memory element and the processor;
estimating a processing time for the processor to process the workload; and
setting at least one of the processing rate characteristic of the processor and the data transfer rate in dependence upon the estimated processing time and access time, and
wherein the method comprises:
calculating an amount of data that is to be transferred between the memory element and the processor at each of a plurality of stages of processing the workload;
determining a plurality of quality of service values to be included in requests to the memory controller based upon the calculated amount of data that is to be transferred at each stage;
sending, by the memory access engine, requests to the memory controller including the determined quality of service values;
transferring data between the memory element and the processor in accordance with the data transfer rate; and
processing, by the processor, the workload in accordance with the processing rate characteristic.

16. A non-transitory computer-readable storage medium storing instructions that, when executed by an information processing apparatus comprising a processor having a processing rate characteristic, a memory element connected to the processor via a data link, and a memory access engine connected to the data link configured to send requests to a memory controller and which information processing apparatus is configured to control a data transfer rate between the memory element and the processor, causes the information processing apparatus to perform a method comprising, in advance of processing a workload:
estimating an access time required to transfer an amount of the workloa

Assignees

• Apical Ltd
• Advanced Risc Mach Ltd

Inventors

• Croxford Daren
• Saeed Sharjeel
• RAO Jayavarapu Srinivasa
• Debattista Aaron

Classifications

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06F9/5038 Primary

  considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups) · CPC title

• G06F3/0604

  Improving or facilitating administration, e.g. storage management · CPC title

• G06F3/0632

  by initialisation or re-initialisation of storage systems · CPC title