Compute optimization mechanism for deep neural networks

Patent metadata

Field	Value
Publication number	US-11348198-B2
Application number	US-202117145885-A
Country	US
Kind code	B2
Filing date	Jan 11, 2021
Priority date	Apr 24, 2017
Publication date	May 31, 2022
Grant date	May 31, 2022

Abstract

An apparatus to facilitate compute optimization is disclosed. The apparatus includes a plurality of processing units each comprising a plurality of execution units (EUs), wherein the plurality of EUs comprise a first EU type and a second EU type.

First claim

What is claimed is:

1. A graphics processing unit comprising one or more multiprocessors, at least one of the one or more multiprocessors including:
a register file to store operands for a plurality of different types of operands; and
a plurality of processing cores, including:
a first set of processing cores of a first type including a first set of floating point units (FPUs) to execute instructions to perform multi-dimensional matrix math operations on a first set of operands in a first set of registers of the register file; and
a second set of processing cores of a second type, the second set of processing cores being different from the first set of processing cores, the second set of processing cores to perform general purpose graphics processing unit (GPGPU) operations on a second set of operands in a second set of registers of the register file, wherein the second set of processing cores comprises:
a set of integer units to execute instructions to perform integer operations; and
a second set of FPUs to execute instructions to perform floating point operations, the second set of FPUs comprising a first subset of FPUs to perform 32-bit floating point (FP32) operations and a second subset of FPUs to perform 64-bit floating point (FP64) operations.

2. The graphics processing unit as in claim 1 , further comprising a thread dispatcher to dispatch threads associated with instructions executed by the graphics processing unit.

3. The graphics processing unit as in claim 1 , wherein matrix operations performed on the first set of operands include FP64 operations.

4. The graphics processing unit as in claim 1 , wherein the instructions to perform the multi-dimensional matrix math operations are associated with primitives provided by a machine learning framework.

5. The graphics processing unit as in claim 1 , wherein the first set of processing cores of the first type is configured to perform an in-place matrix to vector transformation for a first type of operand stored in the register file.

6. The graphics processing unit as in claim 5 , wherein the in-place matrix to vector transformation includes a set of operations having a source and destination, the source and destination within the register file.

7. The graphics processing unit as in claim 6 , wherein the source includes a register address start limit, stride, number of elements, and element size.

8. The graphics processing unit as in claim 1 , wherein the at least one of the one or more multiprocessors further comprises an instruction cache to store a first instruction associated with the first set of operands and a second instruction associated with the second set of operands.

9. The graphics processing unit as in claim 1 , wherein the first set of processing cores of the first type are associated with a first memory channel and the second set of processing cores of the second type are associated with a second memory channel.

10. A method comprising:
receiving a first set of operands from a first set of registers of a register file at a first set of processing cores of a first type at a graphics processing unit;
receiving a second set of operands from a second set of registers of a register file at a second set of processing cores of a second type, the second set of processing cores being different from the first set of processing cores, at the graphics processing unit;
performing multi-dimensional matrix math operations on the first set of operands at the first set of processing cores; and
performing general-purpose graphics processing unit (GPGPU) operations on the second set of operands at the second set of processing cores, wherein performing the GPGPU operations at the second set of processing cores comprises:
executing instructions at a set of floating point units (FPUs), to perform floating point operations, wherein executing instructions at the set of floating point units (FPUs) comprises performing 32-bit floating point (FP32) operations at a first subset of FPUs; and
performing 64-bit floating point (FP64) operations at a second subset of FPUs.

11. The method as in claim 10 , further comprising dispatching threads associated with instructions executed by the graphics processing unit.

12. The method as in claim 11 , wherein performing the GPGPU operations at the second set of processing cores further comprises executing instructions at a set of integer units to perform integer operations.

13. The method as in claim 10 , wherein the first set of processing cores is configured to perform an in-place matrix to vector transformation for a first type of operand stored in the register file.

14. The method as in claim 13 , wherein the in-place matrix to vector transformation
includes a set of operations having a source and destination, the source and destination within the register file and the source includes a register address start limit, stride, number of elements, and element size.

15. A data processing system comprising:
a memory device; and
a graphics processing unit comprising one or more multiprocessors, at least one of the one or more multiprocessors including a register file to store operands for a plurality of different types of operands and a plurality of processing cores, including:
a first set of processing cores of a first type including a first set of floating point units (FPUs) to execute instructions to perform multi-dimensional matrix math operations on a first set of operands in a first set of registers of the register file; and
a second set of processing cores of a second type, the second set of processing cores being different from the first set of processing cores, the second set of processing cores to perform general purpose graphics processing unit (GPGPU) operations on a second set of operands in a second set of registers of the register file, wherein the second set of processing cores comprises:
a set of integer units to execute instructions to perform integer operations; and
a second set of FPUs to execute instructions to perform floating point operations, the second set of FPUs comprising a first subset of FPUs to perform 32-bit floating point (FP32) operations and a second subset of FPUs to perform 64-bit floating point (FP64) operations.

16. The data processing system as in claim 15 , further comprising a thread dispatcher to dispatch threads associated with instructions executed by the graphics processing unit.

17. The data processing system as in claim 15 , wherein matrix operations performed on the first set of operands include FP64 operations.

18. The data processing system as in claim 15 , wherein the instructions to perform the multi-dimensional matrix math operations are associated with primitives provided by a machine learning framework.

19. The data processing system as in claim 15 , wherein the first set of processing cores of the first type is configured to perform an in-place matrix to vector transformation for a first type of operand stored in the register file, the in-place matrix to vector transformation includes a set of operations having a source and destination, the source and destination within the register file, and the source includes a register address start limit, stride, number of elements, and element size.

20. The data processing system in claim 15 , wherein the at least one of the one or more multiprocessors further comprises an instruction cache to store a first instruction associated with the first set of operands and a second instruction associated with the second set of opera

Assignees

• Intel Corp

Inventors

• Surti Prasoonkumar
• Srinivasa Narayan
• Chen Feng
• Ray Joydeep
• Ashbaugh Ben J
• Galoppo Von Borries Nicolas C
• Nurvitadhi Eriko
• Vembu Balaji
• Lin Tsung-Han
• Sinha Kamal
• Barik Rajkishore
• Baghsorkhi Sara S
• Gottschlich Justin E
• Koker Altug
• Satish Nadathur Rajagopalan
• Akhbari Farshad
• Kim Dukhwan
• Fu Wenyin
• Schluessler Travis T
• Mastronarde Josh B
• Hurd Linda L
• Feit John H
• Boles Jeffery S
• Lake Adam T
• Vaidyanathan Karthik
• Burke Devan
• Maiyuran Subramaniam
• Appu Abhishek R

Classifications

• G06N3/045 Primary

  Combinations of networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title