Hybrid analog-digital matrix processors

Patent metadata

Field	Value
Publication number	US-11775779-B2
Application number	US-202117246892-A
Country	US
Kind code	B2
Filing date	May 3, 2021
Priority date	Feb 26, 2019
Publication date	Oct 3, 2023
Grant date	Oct 3, 2023

Abstract

Techniques for computing matrix operations for arbitrarily large matrices on a finite-sized hybrid analog-digital matrix processor are described. Techniques for gain adjustment in a finite-sized hybrid analog-digital matrix processor are described which enable the system to obtain higher energy efficiencies, greater physical density and improved numerical accuracy. In some embodiments, these techniques enable maximization of the predictive accuracy of a GEMM-based convolutional neural network using low-precision data representations.

First claim

What is claimed is:

1. An accelerator comprising:
a plurality of digital-to-analog converters (DACs) configured to generate a plurality of input analog signals based on a data vector;
an analog processor arranged to perform matrix-vector multiplication and comprising inputs and outputs;
a plurality of n-bit analog-to-digital converters (ADCs) coupled to respective outputs of the analog processor, where n is equal to or less than 16;
a plurality of analog amplifiers, coupled between the plurality of DACs and respective inputs of the analog processor, configured to generate a plurality of amplified input analog signals based on the plurality of input analog signals; and
a controller configured to:
receive a set of parameters representing a matrix;
set a first gain of a first analog amplifier of the plurality of analog amplifiers and a second gain of a second amplifier of the plurality of analog amplifiers, wherein the first gain is different from the second gain;
control the analog processor to generate a plurality of output analog signals based on the plurality of amplified input analog signals and the set of parameters; and
control the plurality of ADCs to digitize the plurality of output analog signals.

2. The accelerator of claim 1 , wherein the first and second gains are greater or less than 1.

3. The accelerator of claim 1 , wherein the analog processor is a photonic processor configured to receive light and to perform matrix-vector multiplication using the light.

4. The accelerator of claim 3 , wherein the plurality of analog amplifiers are optical amplifiers.

5. The accelerator of claim 3 , further comprising a plurality of homodyne photodetectors coupled between the outputs of the analog processor and the plurality of n-bit ADCs.

6. The accelerator of claim 1 , where n is equal to or less than 12.

7. A method for performing matrix-vector multiplication, comprising:
generating a plurality of input analog signals based on a data vector;
setting a first gain of a first multiplication unit of a plurality of multiplication units and a second gain of a second multiplication unit of the plurality of multiplication units, wherein the first gain is different from the second gain;
controlling an analog processor based on a set of parameters representing a matrix;
with the plurality of multiplication units, generating a plurality of scaled input analog signals by scaling the plurality of input analog signals;
with the analog processor, generating a plurality of output analog signals based on the plurality of scaled input analog signals and the set of parameters; and
controlling a plurality of n-bit analog-to-digital converters (ADCs), where n is equal to or less than 16, to digitize the plurality of output analog signals.

8. The method of claim 7 , where n is equal to or less than 12.

9. The method of claim 7 , wherein:
the matrix has at least MxM parameters,
the parameters of the set are represented by m1bits,
the data vector has entries represented by m2 bits, and
n is less than log2(M)+m1+m2.

10. The method of claim 7 , wherein the first and second gains are greater or less than 1.

11. The method of claim 7 , wherein m1 is equal to m2.

12. The method of claim 7 , wherein generating the plurality of output analog signals based on the plurality of scaled input analog signals and the set of parameters comprises performing a convolution based on the plurality of scaled input analog signals and the set of parameters.

13. The method of claim 7 , wherein controlling the analog processor comprises:
controlling, based on the set of parameters, the analog processor with a plurality of matrices that, collectively, represent an arbitrary matrix.

14. The method of claim 7 , further comprising determining the first and second gains based on the set of parameters and the data vector.

15. The method of claim 14 , wherein determining the first and second gains comprises determining the first and second gains based on statistical bounds on the set of parameters and statistical bounds on the data vector.

16. The method of claim 14 , wherein determining the first and second gains comprises determining the first and second gains using machine learning and regularizing the first and second gains using a regularization loss term.

17. The method of claim 7 , wherein data vector is represented according to a floating-point binary representation.

18. The method of claim 7 , wherein the set of parameters is represented according to a floating-point binary representation.

19. The method of claim 7 , wherein the scaling the plurality of input analog signals is represented according to a floating-point binary representation.

20. An accelerator comprising:
a plurality of digital-to-analog converters (DACs) configured to generate a plurality of input analog signals based on a data vector;
an analog processor arranged to generate a plurality of output analog signals by performing matrix-vector multiplication based on the plurality of input analog signals, the analog processor comprising inputs and outputs;
a plurality of n-bit analog-to-digital converters (ADCs), where n is equal to or less than 16;
a plurality of analog scaling units, coupled between respective outputs of the analog processor and the plurality of ADCs, configured to generate a plurality of scaled output analog signals based on the plurality of output analog signals; and
a controller configured to:
receive a set of parameters representing a matrix;
set a first gain of a first analog scaling unit of the plurality of analog scaling units and a second gain of a second analog scaling unit of the plurality of analog scaling units, wherein the first gain is different from the second gain; and
control the analog processor based on the set of parameters.

21. The accelerator of claim 20 , wherein the first gain of the first analog scaling unit is greater or less than 1.

22. The accelerator of claim 20 , wherein the analog processor is a photonic processor configured to receive light and to perform matrix-vector multiplication using the light.

23. The accelerator of claim 22 , wherein the plurality of analog scaling units comprises a plurality of optical amplifiers.

24. The accelerator of claim 20 , where n is equal to or less than 12.

25. A method for performing matrix-vector multiplication, comprising:
generating a plurality of input analog signals based on a data vector;
setting a first gain of a first multiplication unit-of a plurality of multiplication units and a second gain of a second multiplication unit of the plurality of multiplication units, wherein the first gain is different from the second gain;
controlling an analog processor based on a set of parameters representing a matrix;
with the analog processor, generating a plurality of output analog signals based on the plurality of input analog signals and the set of parameters;
with the plurality of multiplication units, generating a plurality of scaled output analog signals by scaling the plurality of output analog signals; and
controlling a plurality of n-bit analog-to-digital converters (ADCs), where n is equal to or less than 16, to digitize the plurality of scaled output analog signals.

26. The method of claim 25 , where n is equal to or less than 12.

27. The method of claim 25 , wherein:
the matrix has at least MxM parameters,
the par

Assignees

• Lightmatter Inc

Inventors

• Kenney Tyler J
• Forsythe Martin B Z
• Lazovich Tomo
• Bunandar Darius

Classifications

• G06J1/02 Primary

  Differential analysers · CPC title

• G06F17/16

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06J1/005 Primary

  for correlation; for convolution; for Z or Fourier Transform · CPC title

• G06N3/065

  Analogue means · CPC title

• G06F7/32

  Merging, i.e. combining data contained in ordered sequence on at least two record carriers to produce a single carrier or set of carriers having all the original data in the ordered sequence {merging methods in general}(G06F7/36 takes precedence) · CPC title