Special purpose neural network training chip

Patent metadata

Field	Value
Publication number	US-12524658-B2
Application number	US-202217694598-A
Country	US
Kind code	B2
Filing date	Mar 14, 2022
Priority date	May 17, 2017
Publication date	Jan 13, 2026
Grant date	Jan 13, 2026

Abstract

Methods, systems, and apparatus including a special purpose hardware chip for training neural networks are described. The special-purpose hardware chip may include a scalar processor configured to control computational operation of the special-purpose hardware chip. The chip may also include a vector processor configured to have a 2-dimensional array of vector processing units which all execute the same instruction in a single instruction, multiple-data manner and communicate with each other through load and store instructions of the vector processor. The chip may additionally include a matrix multiply unit that is coupled to the vector processor configured to multiply at least one two-dimensional matrix with a second one-dimensional vector or two-dimensional matrix in order to obtain a multiplication result.

First claim

What is claimed is:

1 . A special-purpose hardware chip for training neural networks, the special-purpose hardware chip comprising:
a scalar processor configured to control computational operation of the special-purpose hardware chip; a vector processor having a 2-dimensional array of vector processing units; and a matrix multiply unit that is coupled to the vector processor and configured to multiply at least a first two-dimensional matrix with a first one-dimensional vector or a second two-dimensional matrix in order to obtain a multiplication result, wherein the vector processor includes a plurality of lanes, wherein each vector processing unit of the 2-dimensional array of vector processing units in the vector processor is located in a respective lane of the plurality of lanes, wherein one or more vector processing units of the 2-dimensional array of vector processing units that are located in the same lane are configured to communicate with one another through respective load and store instructions.

2 . The special-purpose hardware chip of claim 1 , further comprising:
a vector memory configured to provide private memory to the vector processor.

3 . The special-purpose hardware chip of claim 1 , further comprising:
a scalar memory configured to provide private memory to the scalar processor.

4 . The special-purpose hardware chip of claim 1 , further comprising:
a transpose unit configured to perform a transposition operation of a matrix.

5 . The special-purpose hardware chip of claim 1 , further comprising:
a reduction and permutation unit configured to perform a reduction on numbers and permute the numbers among different lanes of a vector array.

6 . The special-purpose hardware chip of claim 1 , further comprising:
a first memory configured to store data of the special-purpose hardware chip.

7 . The special-purpose hardware chip of claim 1 , further comprising a sparse computation core.

8 . The special-purpose hardware chip of claim 1 , further comprising:
an interface; and an inter-chip interconnect, which connects the interface or resources on the special purpose hardware chip to other special-purpose hardware chips or resources.

9 . The special-purpose hardware chip of claim 8 , further comprising:
a first memory, wherein the inter-chip interconnect connects the interface and the first memory to other special-purpose hardware chips.

10 . The special-purpose hardware chip of claim 8 , wherein the interface is a host interface to a host computer.

11 . The special-purpose hardware chip of claim 8 , wherein the interface is a standard network interface to a network of host computers.

12 . The special-purpose hardware chip of claim 8 , further comprising a scalar memory and a vector memory.

13 . The special-purpose hardware chip of claim 8 , wherein a scalar instruction set of the instructions includes arithmetic operations used in address calculations, load/store instructions, and branch instructions, and wherein the remaining instructions of the instructions encode instructions for the vector processor and the matrix multiply unit.

14 . The special-purpose hardware chip of claim 1 , wherein each vector processing unit in the 2-dimensional array of vector processing units includes 32 registers.

15 . The special-purpose hardware chip of claim 1 , wherein each vector processing unit in the 2-dimensional array of vector processing units is configured to perform at least one of a floating point operation or integer operation.

16 . The special-purpose hardware chip of claim 1 , wherein each vector processing unit in the 2-dimensional array of vector processing units is configured to execute two respective arithmetic logic unit (ALU) instructions, a respective load instruction, and a respective store instruction in each clock cycle.

17 . The special-purpose hardware chip of claim 16 , wherein each vector processing unit in the 2-dimensional array of vector processing units is configured to compute respective offset memory addresses for executing the respective load and store instructions in each clock cycle.

Assignees

• Google Llc

Inventors

• Norrie Thomas
• Temam Olivier
• Phelps Andrew Everett
• Jouppi Norman Paul

Classifications

• G06N3/08

  Learning methods · CPC title

• G06F17/16 Primary

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06F9/3885

  using a plurality of independent parallel functional units · CPC title