Electronic device and method for inference binary and ternary neural networks

Patent metadata

Field	Value
Publication number	US-12039430-B2
Application number	US-202017098589-A
Country	US
Kind code	B2
Filing date	Nov 16, 2020
Priority date	Nov 15, 2019
Publication date	Jul 16, 2024
Grant date	Jul 16, 2024

Abstract

A method for computing an inner product on a binary data, a ternary data, a non-binary data, and a non-ternary data using an electronic device. The method includes calculating the inner product on a ternary data, designing a fused bitwise data path to support the inner product calculation on the binary data and the ternary data, designing a FPL data path to calculate an inner product between one of the non-binary data and the non-ternary data and one of the binary data and the ternary data, and distributing the inner product calculation for the binary data and the ternary data and the inner product between one of the non-binary data and the non-ternary data and one of the binary data and the ternary data in the fused bitwise data path and the FPL data path.

First claim

What is claimed is:

1. A method for computing an inner product on a binary data, a ternary data, a non-binary data, and a non-ternary data, comprising:
calculating, by an electronic device, the inner product on the ternary data;
designing, by the electronic device, a fused bitwise data path to support the inner product calculation on the binary data and the ternary data;
designing, by the electronic device, a Full Precision Layer (FPL) data path to calculate an inner product between one of the non-binary data and the non-ternary data and one of the binary data and the ternary data; and
distributing, by the electronic device, the inner product calculation for the binary data and the ternary data and the inner product between one of the non-binary data and the non-ternary data and one of the binary data and the ternary data respectively in the fused bitwise data path and the FPL data path,
wherein the fused bitwise data path is configured to support the inner product calculation for the binary data and the inner product calculation for the ternary data.

2. The method as claimed in claim 1 , wherein designing the fused bitwise data path to support the inner product calculation for the binary data and the ternary data comprises:
receiving, by the electronic device, one of the ternary data and the binary data;
determining, by the electronic device, a mode of operation for the ternary data and the binary data;
processing, by the electronic device, at least one of the ternary data and the binary data in a XNOR gate and a AND gate using at least one popcount logic based on the determined mode of the operation;
receiving, by the electronic device, at least one of the processed ternary data and the processed binary data to an accumulator; and
generating, by the electronic device, at least one of a final ternary value and a final binary value.

3. The method as claimed in claim 2 , further comprising processing, by the electronic device, at least one of the ternary data and the binary data in the XNOR gate and the AND gate using the at least one popcount logic based on the determined mode of the operation, when a fused data path engine of the electronic device is configured to support the binary data used by a Binary Neural Network (BNN) model and the ternary data used by a Ternary Neural Network (TNN) model;
receiving one bit of a first vector and one bit of a second vector, using a first XNOR gate, to generate a product vector of length N bits as an output of the first XNOR gate;
receiving the one bit of the first vector and the one bit of the second vector, using a first AND gate, to generate a product vector of length N bits as an output of the first AND gate;
receiving one bit of a third vector and one bit of a fourth vector, using a second XNOR gate, to generate a product vector of length N bits as an output of the second XNOR gate;
receiving the one bit of the third vector and the one bit of the fourth vector, using a second AND gate, to generate a product vector of length N bits as an output of the second AND gate;
feeding the output of the first XNOR gate and the output of the first AND gate as an input of a first multiplexer, wherein an output of the first multiplexer comprises of a mask vector of a resultant inner product between two ternary vectors in case of the ternary data or the resultant inner product vector of a first binary vector pair in case of the binary data;
feeding the output of the first AND gate and the output of the second XNOR gate as an input of a third AND gate;
receiving an input from the output of the second XNOR gate, the output of the second AND gate, and an output of the third AND gate using a second multiplexer, wherein an output of the second multiplexer comprises a value vector of the resultant inner product of the two ternary vector pairs which is only affected by non-zero element pairs from the value vector of the input ternary vector pairs in case of the ternary data, wherein the output of second multiplexer comprises a resultant inner product vector of a second binary vector pair in case of the binary data;
receiving a first bit length and the input from the first multiplexer through a first pop-counter, wherein the output of the first multiplexer feeds as an input of the first pop-counter, wherein the first pop-counter computes the number of 1 s in the resultant mask vector in case of the ternary data and the number of 1s in the resultant inner product vector in case of the binary data which is forwarded to the third multiplexer, in case of the binary data and forwarded to a fourth multiplexer, in case of the ternary data; and
receiving the second bit length, the output from the first multiplexer and a second pop-counter to the fourth multiplexer, wherein the second pop-counter computes a number of one value from the output of the second multiplexer that is forwarded to the fourth multiplexer, wherein an output of the second pop-counter is left shifted by one value, wherein the output of fourth multiplexer comprises either the output from the first pop-counter in case of the ternary data or the second bit length in case of the binary data type B or the output from the second pop-counter in case of the binary data type A,
wherein the left shifted output of the first pop-counter and the output of the third multiplexer are subtracted in a first subtractor, wherein an output of the first subtractor denotes the inner product value of the first binary vector pair,
wherein the left shifted output of the second pop-counter and the output of the fourth multiplexer are subtracted in a second subtractor, wherein an output of the second subtractor indicates the inner product value between two ternary vector pair, in case of the ternary data or the inner product value between the second binary vector pair in case of the binary data,
wherein the output of the second subtractor and the output of the first subtractor are added in a first adder in case of the binary data, wherein the output of a fifth multiplexer selects either the output from the first adder in case of the binary data, or the output from the second subtractor in case of the ternary data, wherein the output of the fifth multiplexer is added with a first accumulator using a second adder, and
wherein an output of the second adder is stored in the first accumulator, wherein the output of the second adder is compared with a threshold in a comparator to generate an output value.

4. The method as claimed in claim 3 , further comprising processing, by the electronic device, at least one of the ternary data and the binary data in the XNOR gate and the AND gate using the at least one popcount logic based on the determined mode of the operation, when the fused data path engine of the electronic device is configured to support the ternary data used by TNN model;
receiving one bit of a first vector and one bit of a second vector, using the first AND gate, to generate a product vector of length N bits as an output of the first AND gate;
receiving one bit of a third vector and one bit of a fourth vector using the second XNOR gate to generate a product vector of length N bits as an output of the second XNOR gate;
feeding the output of the first AND gate as an input of the first multiplexer;
feeding the output of the first AND gate and the output of the second XNOR gate as an input of the third AND gate;
receiving the input from the output of the second XNOR gate, and an output of the third AND gate using the second multiplexer, wherein an output of the second multiplexer comprises elements in a resultant bit vector obtained from bitwise operation between two ternary vectors; and
receiving a second bit length and an input from the second multiplexer through the second pop-counter and the input from the first multiplexer through the first pop-counte

Assignees

• Samsung Electronics Co Ltd

Inventors

• Roy Arnab
• Das Saptarsi
• Deshwal Ankur
• Chandrasek Haran Kiran Kolar
• Lee Sehwan

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06F7/5443 Primary

  Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00) · CPC title

• G06N3/045 Primary

  Combinations of networks · CPC title

• G06N3/063

  using electronic means · CPC title