Enhancing processing performance of a DNN module by bandwidth control of fabric interface

Patent metadata

Field	Value
Publication number	US-10628345-B2
Application number	US-201815950644-A
Country	US
Kind code	B2
Filing date	Apr 11, 2018
Priority date	Apr 17, 2017
Publication date	Apr 21, 2020
Grant date	Apr 21, 2020

Abstract

An exemplary computing environment having a DNN module can maintain one or more bandwidth throttling mechanisms. Illustratively, a first throttling mechanism can specify the number of cycles to wait between transactions on a cooperating fabric component (e.g., data bus). Illustratively, a second throttling mechanism can be a transaction count limiter that operatively sets a threshold of a number of transactions to be processed during a given transaction sequence and limits the number of transactions such as multiple transactions in flight to not exceed the set threshold. In an illustrative operation, in executing these two exemplary calculated throttling parameters, the average bandwidth usage and the peak bandwidth usage can be limited. Operatively, with this fabric bandwidth control, the processing units of the DNN are optimized to process data across each transaction cycle resulting in enhanced processing and lower power consumption.

First claim

What is claimed is:

1. A system for enhanced data processing in a computing environment, the system comprising:
at least one neural network processor;
at least one memory component; and
at least one memory in communication with the at least one neural network processor, the at least one memory having computer-readable instructions stored thereupon that, when executed by the at least one processor, cause the at least one processor to:
receive one or more processing parameters from a cooperating component of the computing environment representative of processing and memory resources required to process input data;
calculate the total available memory on a cooperating fabric component of the computing environment to determine a load parameter for the computing environment;
calculate one or more throttling parameters using the received one or more processing parameters and the calculated load parameter;
select one or more of the calculated throttling parameters for execution; and
throttle one or more cooperating memory components of the computing environment by executing the selected one or more calculated throttling parameters.

2. The system of claim 1 , wherein the throttling is performed by a cooperating DMA component of the computing environment.

3. The system of claim 2 , wherein the calculated throttling parameters comprise transaction limits and gap insertion.

4. The system of claim 3 , wherein the computer-readable instructions further cause the at least one processor to configure a cooperating configurable register on the one or more cooperating memory components according to the calculated throttling parameters.

5. The system of claim 4 , wherein the number of transactions being processed by the exemplary neural network processor of the computing environment are limited to a threshold below the maximum bandwidth of one or more of the cooperating memory components.

6. The system of claim 5 , wherein cooperating memory component comprises an external fabric component.

7. The system of claim 6 , wherein the limited number of transactions being processed by the exemplary neural network processor according to a selected threshold comprising a specified number of outstanding transactions and a specified number of cycles between transactions.

8. A computer-implemented method, comprising:
receiving one or more processing parameters from a cooperating component of the computing environment representative of processing and memory resources required to process input data;
calculating the total available memory on a cooperating fabric component of the computing environment to determine a load parameter for the computing environment;
calculating one or more throttling parameters using the received one or more processing parameters and the calculated load parameter;
selecting one or more of the calculated throttling parameters for execution; and
throttling one or more cooperating memory components of the computing environment by executing on the one or more cooperating memory components the selected one or more calculated throttling parameters, the parameters comprising inserting a gap between transactions processed by a neural network processor of an exemplary computing environment and limiting the number of transactions processed by a neural network processor of an exemplary computing environment.

9. The computer-implemented method of claim 8 , further comprising receiving instructions from a control computer processing unit capable of executing the calculated throttling parameters.

10. The computer-implemented method of claim 8 , further comprising configuring a configurable register with data elements from the calculated throttling parameters for use in performing throttling.

11. The computer-implemented method of claim 8 , further comprising setting a transaction threshold parameter value comprising a specified number of outstanding transactions and a specified number of cycles between transactions for use in throttling.

12. The computer-implemented method of claim 11 , further comprising utilizing the set transaction threshold parameter by a neural network processor of a computing environment during throttling.

13. The computer-implemented method of claim 8 , further comprising selecting one or more memory components on which to perform throttling.

14. The computer-implemented method of claim 13 , further comprising providing one or more throttling instructions to an external fabric component comprising the calculated throttling parameters.

15. The computer-implemented method of claim 14 , further comprising communicating with one or more memory management components of a computing environment comprising fabric clients to affect one or more operations comprising quality of service operations, hardware quiesce operations, and hardware debugging operations.

16. A computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by one or more processors of a computing device, cause the one or more processors of the computing device to:
receive one or more processing parameters from a cooperating component of the computing environment representative of processing and memory resources required to process input data;
calculate the total available memory on a cooperating fabric component of the computing environment to determine a load parameter for the computing environment;
calculate one or more throttling parameters using the received one or more processing parameters and the calculated load parameter;
select one or more of the calculated throttling parameters for execution; and
throttle one or more cooperating memory components of the computing environment by executing the selected one or more calculated throttling parameters.

17. The computer-readable storage medium of claim 16 , wherein the instructions further cause the one or more processors of the computing device to:
configure a configurable register resident on one or more cooperating memory components with data elements from the calculated throttling parameters for use in performing throttling.

18. The computer-readable storage medium of claim 17 , wherein the instructions further cause the one or more processors of the computing device to:
receive one or more instructions from a control computer processing unit capable of executing the calculated throttling parameters.

19. The computer-readable storage medium of claim 16 , wherein the instructions further cause the one or more processors of the computing device to:
set a transaction threshold parameter value comprising a specified number of outstanding transactions and a specified number of cycles between transactions for use in throttling.

20. The computer-readable storage medium of claim 16 , wherein the instructions further cause the one or more processors of the computing device to:
select one or more memory components on which to perform throttling.

21. The computer readable medium of claim 16 , wherein the memory component cooperates with a physical sensor capable of producing input data comprising audio data, video data, haptic sensory data, and other data for subsequent processing by the one or more cooperating processing units.

22. The computer readable medium of claim 21 , wherein the cooperating processing units electronically cooperate with one or more output physical components operative to receive for human interaction processed input data comprising audio data, video data, haptic sensory d

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Mcbride Chad Balling
• Heil Timothy Hume
• Ambardekar Amol Ashok
• Petre George
• Cedola Kent D
• Wall Larry Marvin
• Bobrov Boris

Classifications

• G06F1/3275

  Power saving in memory, e.g. RAM, cache · CPC title

• G06F9/5077

  Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455) · CPC title

• G06F15/8007

  single instruction multiple data [SIMD] multiprocessors · CPC title

• G06N3/06

  Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons · CPC title

• G06F2209/485

  Resource constraint · CPC title