Compute optimization mechanism for deep neural networks

Patent metadata

Field	Value
Publication number	US-12086705-B2
Application number	US-201715858014-A
Country	US
Kind code	B2
Filing date	Dec 29, 2017
Priority date	Dec 29, 2017
Publication date	Sep 10, 2024
Grant date	Sep 10, 2024

Abstract

An apparatus to facilitate compute optimization is disclosed. The apparatus includes a at least one processor to perform operations to implement a neural network and compute logic to accelerate neural network computations.

First claim

What is claimed is:

1. An apparatus, comprising:
at least one processor to perform operations to implement a neural network; and
a graphics processing unit (GPU) including circuitry configured to accelerate neural network computations, the circuitry comprising:
processing circuitry configured to perform general-purpose graphics computations, the processing circuitry including a single instruction multiple thread (SIMT) architecture;
a local memory to store one or more graph representations associated with a neural network, the one or more graph representations to indicate node adjacency for the neural network;
a graph processing unit (GrPU) including instruction execution circuitry configured to accelerate computations on the one or more graph representations in response to a request from the processing circuitry, wherein the GrPU includes multiple single instruction multiple data (SIMD) hardware threads to concurrently traverse multiple graph representations and execute instructions associated with the multiple graph representations;
a compilation unit (CU) including instruction execution circuitry configured to dynamically compile shader kernels locally on the GPU; and
wherein the GrPU is configured to perform a compute operation implemented via a dynamically compiled shader, the dynamically compiled shader is dynamically compiled by the CU and executed by the GrPU in response to a condition detected by the GPU, the condition associated with input data of a neural network computation.

2. The apparatus of claim 1 , wherein the GPU is configured to perform non-uniform quantization for the neural network and the one or more graph representations include an adjacency list or an adjacency matrix.

3. The apparatus of claim 1 , wherein the GPU includes circuitry to accelerate application of an activation function for an operation associated with the neural network.

4. The apparatus of claim 3 , wherein the GPU includes:
circuitry to provide a fetch stage to receive input values;
circuitry to provide an execute stage to perform computation operations on the input values; and
circuitry to provide a writeback stage to pack and prepare results to be output.

5. The apparatus of claim 4 , wherein the fetch stage is configured to analyze and identify a first operation to be implemented via first execute stage circuitry and a second operation to be implemented via second execution stage circuitry.

6. The apparatus of claim 5 , wherein the execute stage comprises:
first execute stage circuitry to implement a first set of activation functions; and
second execute stage circuitry to implement a second set of activation functions.

7. The apparatus of claim 6 , wherein the writeback stage is configured to:
receive results from the first execute stage circuitry and the second execute stage circuitry; and
output the results in a format associated with an output tensor.

8. The apparatus of claim 1 , wherein the circuitry of the GPU is configured to:
detect the condition associated with input data of a neural network computation;
based on the condition, compile a modified shader that is to configure the GPU to perform a modified neural network computation; and
perform the modified neural network computation via the compiled modified shader.

9. A data processing system comprising:
a memory device to store instructions; and
a heterogeneous processor comprising a central processing unit (CPU) core, a graphics processing unit (GPU) core, and a compute accelerator core, the heterogeneous processor configured to accelerate neural network computations, wherein the heterogeneous processor includes circuitry comprising:
a processing circuitry configured to perform general-purpose graphics computations, the processing circuitry including a single instruction multiple thread (SIMT) architecture;
a local memory to store one or more graph representations, the one or more graph representations to indicate node adjacency for the neural network;
a graph processing unit (GrPU) including instruction execution circuitry configured to accelerate computations on the one or more graph representations in response to a request from the processing circuitry, wherein the GrPU includes multiple single instruction multiple data (SIMD) hardware threads to concurrently traverse multiple graph representations and execute instructions associated with the multiple graph representations;
a compilation unit (CU) including instruction execution circuitry configured to dynamically compile shader kernels locally on the GPU; and
wherein the GrPU is configured to perform a compute operation implemented via a dynamically compiled shader and the dynamically compiled shader is dynamically compiled by the CU and executed by the GrPU in response to a condition detected by the GPU, the condition associated with input data of a neural network computation.

10. The data processing system of claim 9 , wherein the heterogeneous processor is configured to process an input image via the neural network, wherein to process the input image includes to:
crop the input image into a plurality of planar segments having a configurable overlap ratio, each of the plurality of planar segments having a single color channel;
cluster the planar segments into two or more image batches; and
process the two or more image batches via the heterogeneous processor.

11. The data processing system of claim 10 , wherein the heterogeneous processor is configured to process the two or more image batches in parallel via two or more cores.

12. The data processing system of claim 11 , wherein the heterogeneous processor is configured to:
process a first image batch via the CPU core;
process a second image batch via the GPU core; and
process a third image batch via the accelerator core.

13. The data processing system of claim 9 , wherein at least one core of the heterogeneous processor is configured to perform non-uniform quantization for the neural network and the one or more graph representations include an adjacency list or an adjacency matrix.

14. The data processing system of claim 9 , wherein at least one core of the heterogeneous processor includes circuitry to accelerate application of an activation function for an operation associated with the neural network.

15. The data processing system of claim 14 , wherein at least one core of the heterogeneous processor includes:
circuitry to provide a fetch stage to receive input values;
circuitry to provide an execute stage to perform computation operations on the input values; and
circuitry to provide a writeback stage to pack and prepare results to be output.

16. The data processing system of claim 15 , wherein the fetch stage is configured to analyze and identify a first operation to be implemented via first execute stage circuitry and a second operation to be implemented via second execution stage circuitry.

17. The data processing system of claim 16 , wherein the execute stage comprises:
first execute stage circuitry to implement a first set of activation functions; and
second execute stage circuitry to implement a second set of activation functions, and
wherein the writeback stage is configured to:
receive results from the first execute stage circuitry and the second execute stage circuitry; and
output the results in a format associated with an output tensor.

18. A method comprising:
accelerating, in response to a request from general-purpose graphics processing circuitry including a single instruction multiple thread (SIMT) architecture, computa

Assignees

• Intel Corp

Inventors

• Bleiweiss Amit
• Venkatesh Abhishek
• Keskin Gokce
• Gierach John
• Elibol Oguz
• Bar-On Tomer
• Abidi Huma
• Burke Devan
• Menon Jaikrishnan
• Nurvitadhi Eriko
• Thorehosur Appajigowda Pruthvi Gowda
• Schluessler Travis T
• Srivastava Dhawal
• Patel Nishant
• Thomas Anil

Classifications

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/098

  Distributed learning, e.g. federated learning · CPC title