Special purpose neural network training chip

Patent metadata

Field	Value
Publication number	US-11275992-B2
Application number	US-201815983056-A
Country	US
Kind code	B2
Filing date	May 17, 2018
Priority date	May 17, 2017
Publication date	Mar 15, 2022
Grant date	Mar 15, 2022

Abstract

Methods, systems, and apparatus including a special purpose hardware chip for training neural networks are described. The special-purpose hardware chip may include a scalar processor configured to control computational operation of the special-purpose hardware chip. The chip may also include a vector processor configured to have a 2-dimensional array of vector processing units which all execute the same instruction in a single instruction, multiple-data manner and communicate with each other through load and store instructions of the vector processor. The chip may additionally include a matrix multiply unit that is coupled to the vector processor configured to multiply at least one two-dimensional matrix with a second one-dimensional vector or two-dimensional matrix in order to obtain a multiplication result.

First claim

What is claimed is:

1. A special-purpose hardware chip for training neural networks, the special-purpose hardware chip comprising:
a scalar processor configured to control computational operation of the special-purpose hardware chip;
a vector processor configured to have a 2-dimensional array of vector processing units which all execute the same instruction in a single instruction, multiple-data manner and communicate with each other through load and store instructions of the vector processor;
a matrix multiply unit that is coupled to the vector processor configured to multiply at least one two-dimensional matrix with a second one-dimensional vector or two-dimensional matrix in order to obtain a multiplication result, and
a reduction and permutation unit configured to perform a reduction on numbers and permute the numbers among different lanes of the 2-dimensional array.

2. The special-purpose hardware chip of claim 1 , further comprising:
a vector memory configured to provide private memory to the vector processor.

3. The special-purpose hardware chip of claim 1 , further comprising:
a scalar memory configured to provide private memory to the scalar processor.

4. The special-purpose hardware chip of claim 1 , further comprising:
a transpose unit configured to perform a transposition operation of a matrix.

5. The special-purpose hardware chip of claim 1 , further comprising:
a high-bandwidth memory configured to store data of the special-purpose hardware chip.

6. The special-purpose hardware chip of claim 1 , further comprising a sparse computation core.

7. The special-purpose hardware chip of claim 1 , further comprising:
an interface; and
an inter-chip interconnect, which connects the interface or resources on the special-purpose hardware chip to other special-purpose hardware chips or resources.

8. The special-purpose hardware chip of claim 7 , further comprising:
a plurality of high bandwidth memory; wherein the inter-chip interconnect connects the interface and one or more of the plurality of high bandwidth memory to other special-purpose hardware chips.

9. The special-purpose hardware chip of claim 7 , wherein the interface is a host interface to a host computer.

10. The special-purpose hardware chip of claim 7 , wherein the interface is a standard network interface to a network of host computers.

11. The special-purpose hardware chip of claim 7 , comprising:
a scalar memory, a vector memory, said scalar processor, said vector processor, and said matrix multiply unit, wherein said scalar processor performs very long instruction word (VLIW) instruction fetch/execute loop and controls said special-purpose hardware chip, wherein after fetching and decoding an instruction bundle, said scalar processor itself only executes instructions found in scalar slots of the instruction bundle using multiple, multi-bit registers of the scalar processor and scalar memory, wherein a scalar instruction set of the instructions found in the scalar slots includes arithmetic operations used in address calculations, load/store instructions, and branch instructions, and wherein the remaining instruction slots encode instructions for the vector processor and said matrix multiply unit.

Assignees

• Google Llc

Inventors

• Norrie Thomas
• Temam Olivier
• Phelps Andrew Everett
• Jouppi Norman Paul

Classifications

• G06N3/08

  Learning methods · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F9/3885

  using a plurality of independent parallel functional units · CPC title

• G06F9/3887

  controlled by a single instruction for multiple data lanes [SIMD] · CPC title

• G06F9/30036

  Instructions to perform operations on packed data, e.g. vector, tile or matrix operations · CPC title