Compute optimization mechanism for deep neural networks

Patent metadata

Field	Value
Publication number	US-11922535-B2
Application number	US-202318168207-A
Country	US
Kind code	B2
Filing date	Feb 13, 2023
Priority date	Apr 24, 2017
Publication date	Mar 5, 2024
Grant date	Mar 5, 2024

Abstract

Embodiments provide mechanisms to facilitate compute operations for deep neural networks. One embodiment comprises a graphics processing unit comprising one or more multiprocessors, at least one of the one or more multiprocessors including a register file to store a plurality of different types of operands and a plurality of processing cores. The plurality of processing cores includes a first set of processing cores of a first type and a second set of processing cores of a second type. The first set of processing cores are associated with a first memory channel and the second set of processing cores are associated with a second memory channel.

First claim

What is claimed is:

1. A graphics processing unit comprising one or more multiprocessors, at least one of the one or more multiprocessors including:
a register file to store a plurality of different types of operands; and
a plurality of processing cores, including:
a first set of processing cores of a first type to perform multi-dimensional matrix operations on a first set of operands in a first set of registers of the register file, wherein the first set of processing cores of the first type includes circuitry to execute instructions to perform matrix operations on the first set of operands in the first set of registers of the register file and the first set of processing cores of the first type are associated with a first memory channel of a memory device coupled with the at least one of the one or more multiprocessors; and
a second set of processing cores of a second type, the second set of processing cores being different from the first set of processing cores, the second set of processing cores to perform general purpose graphics processing unit (GPGPU) operations on a second set of operands in a second set of registers of the register file, wherein the second set of processing cores of the second type are associated with a second memory channel of the memory device coupled with the at least one of the one or more multiprocessors, the second memory channel is distinct from the first memory channel, and the memory device is external to the at least one of the one or more multiprocessors.

2. The graphics processing unit as in claim 1 , wherein the second set of processing cores comprises:
a set of floating point units (FPUs) to execute instructions to perform floating point operations, the set of FPUs to perform 32-bit floating point (FP32) operations and 16-bit floating point (FP16) operations; and
a set of integer units to execute instructions to perform integer operations.

3. The graphics processing unit as in claim 2 , wherein the set of FPUs includes first FPUs to perform the FP32 operations and second FPUs to perform the FP16 operations.

4. The graphics processing unit as in claim 1 , wherein the first set of operands include one or more 64-bit operands.

5. The graphics processing unit as in claim 1 , wherein the first set of processing cores of the first type is configured to perform an in-place matrix to vector transformation for a first type of operand stored in the register file, the in-place matrix to vector transformation includes a set of operations having a source and destination, the source and destination within the register file, wherein the source includes a register address start limit, stride, number of elements, and element size.

6. The graphics processing unit as in claim 1 , wherein the one or more multiprocessors have a single instruction multiple thread (SIMT) architecture.

7. A method to facilitate processing of data at a graphics processing unit (GPU) including one or more multiprocessors, the method comprising:
receiving, at a first set of processing cores of a first type, a first set of operands from first registers of a register file, wherein the first set of processing cores of the first type includes circuitry to execute instructions to perform matrix operations on the first set of operands in the first set of registers of the register file;
receiving, at a second set of processing cores of a second type, a second set of operands from second registers of the register file, the second set of processing cores being different from the first set of processing cores;
performing multi-dimensional matrix math operations on the first set of operands at the first set of processing cores of the first type, wherein the first set of processing cores of the first type are associated with a first memory channel of a memory device coupled with at least one of the one or more multiprocessors; and
performing general-purpose graphics processing unit (GPGPU) operations on the second set of operands at the second set of processing cores, wherein the second set of processing cores of the second type are associated with a second memory channel of the memory device coupled with the at least one of the one or more multiprocessors, the second memory channel is distinct from the first memory channel, and the memory device is external to the at least one of the one or more multiprocessors.

8. The method as in claim 7 , wherein performing the GPGPU operations at the second set of processing cores comprises:
executing instructions at a set of floating point units (FPUs) to perform floating point operations, wherein executing the instructions at the set of FPUs comprises performing 32-bit floating point (FP32) operations and 16 -bit floating point (FP16) operations; and
executing instructions at a set of integer units to perform integer operations.

9. The method as in claim 8 , further comprising performing the FP32 operations at first FPUs of the set of FPUs and performing the FP16 operations at second FPUs of the set of FPUs.

10. The method as in claim 7 , wherein the first set of operands include one or more 64-bit operands.

11. The method as in claim 7 , further comprising performing an in-place matrix to vector transformation for a first type of operand stored in the register file via the first set of processing cores of the first type, wherein the in-place matrix to vector transformation includes a set of operations having a source and destination, the source and destination are within the register file, and the source includes a register address start limit, stride, number of elements, and element size.

12. The method as in claim 7 , wherein the GPU includes one or more multiprocessors comprising the first set of processing cores of the first type, the second set of processing cores of the second type, and an instruction cache to store a first instruction associated with the first set of operands and a second instruction associated with the second set of operands, wherein the one or more multiprocessors have a single instruction multiple thread (SIMT) architecture.

13. A graphics processing system comprising:
a graphics processing unit comprising one or more multiprocessors having a single instruction multiple thread (SIMT) architecture, at least one of the one or more multiprocessors including a register file to store a plurality of different types of operands and a plurality of processing cores, including:
a first set of processing cores of a first type to perform multi-dimensional matrix operations on a first set of operands in a first set of registers of the register file, wherein the first set of processing cores of the first type includes circuitry to execute instructions to perform matrix operations on the first set of operands in the first set of registers of the register file and the first set of processing cores of the first type are associated with a first memory channel of a memory device coupled with the at least one of the one or more multiprocessors; and
a second set of processing cores of a second type, the second set of processing cores being different from the first set of processing cores, the second set of processing cores to perform general purpose graphics processing unit (GPGPU) operations on a second set of operands in a second set of registers of the register file, wherein the second set of processing cores of the second type are associated with a second memory channel of the memory device coupled with the at least one of the one or more multiprocessors, the second memory channel is distinct from the first memory channel, and the memory device is external to the at least one of the one or more multiprocessors.

Assignees

• Intel Corp

Inventors

• Surti Prasoonkumar
• Srinivasa Narayan
• Chen Feng
• Ray Joydeep
• Ashbaugh Ben J
• Galoppo Von Borries Nicolas C
• Nurvitadhi Eriko
• Vembu Balaji
• Lin Tsung-Han
• Sinha Kamal
• Barik Rajkishore
• Baghsorkhi Sara S
• Gottschlich Justin E
• Koker Altug
• Satish Nadathur Rajagopalan
• Akhbari Farshad
• Kim Dukhwan
• Fu Wenyin
• Schluessler Travis T
• Mastronarde Josh B
• Hurd Linda L
• Feit John H
• Boles Jeffery S
• Lake Adam T
• Vaidyanathan Karthik
• Burke Devan
• Maiyuran Subramaniam
• Appu Abhishek R

Classifications

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/098

  Distributed learning, e.g. federated learning · CPC title