Multiplication-free approximation for neural networks and sparse coding

Patent metadata

Field	Value
Publication number	US-11714977-B2
Application number	US-202117554255-A
Country	US
Kind code	B2
Filing date	Dec 17, 2021
Priority date	Jun 29, 2016
Publication date	Aug 1, 2023
Grant date	Aug 1, 2023

Abstract

Systems, apparatuses and methods may provide for replacing floating point matrix multiplication operations with an approximation algorithm or computation in applications that involve sparse codes and neural networks. The system may replace floating point matrix multiplication operations in sparse code applications and neural network applications with an approximation computation that applies an equivalent number of addition and/or subtraction operations.

First claim

We claim:

1. A method comprising:
determining a similarity between two unit vectors based on one or more matrix-vector multiplication operations executed on the two unit vectors; and
replacing the one or more matrix-vector multiplication operations executed on the two unit vectors with a multiplication-free approximation computation.

2. The method of claim 1 , wherein the one or more matrix-vector multiplication operations executed on the two unit vectors are to be replaced with the approximation computation in sparse code applications and neural network applications.

3. The method of claim 2 , wherein the approximation computation is to use a set of basis vectors as an input, and output one or more of a best-matching neuron of the neural network applications or a dictionary atom for the sparse code application that best corresponds to the input basis vectors.

4. The method of claim 3 , wherein a matching pursuit (MP) orthogonal matching pursuit (OMP) computation is to be executed to compute the sparse codes.

5. The method of claim 2 , wherein the one or more matrix-vector multiplication operations are to be replaced with a convolutional filter computation that is a function of a constant and a sub-region of a vector.

6. The method of claim 1 , wherein the one or more matrix-vector multiplication operations are to be replaced by an equivalent number of addition or subtraction operations.

Assignees

• Intel Corp

Inventors

• Chinya Gautham
• Ji Shihao
• Paul Arnab

Classifications

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06K7/10722 Primary

  Photodetector array or CCD scanning · CPC title

• G06F7/483 Primary

  Computations with numbers represented by a non-linear combination of denominational numbers, e.g. rational numbers, logarithmic number system or floating-point numbers {(G06F7/4806, G06F7/4824, G06F7/49, G06F7/491, G06F7/544 take precedence)} · CPC title

• G06F7/487

  Multiplying; Dividing {(G06F7/4833, G06F7/4836 take precedence)} · CPC title