Matrix multiplier and operation method of matrix multiply device including the same

Patent metadata

Field	Value
Publication number	US-2025139193-A1
Application number	US-202418820372-A
Country	US
Kind code	A1
Filing date	Aug 30, 2024
Priority date	Oct 25, 2023
Publication date	May 1, 2025
Grant date	—

Abstract

A matrix multiplier includes an input vector scaler generating a first quantization scaled input vector based on a first input vector, a plurality of common scale coefficients, and first-to-Rth multiplication scale coefficients, a first data type converter generating a first fixed point quantization scaled input vector based on the first quantization scaled input vector, an element array comprising a first processing element generating a first fixed point output element based on the first fixed point quantization scaled input vector and first plurality of quantization sign bits, and a second processing element generating a second fixed point output element based on the first fixed point quantization scaled input vector and second plurality of quantization sign bits, and a second data type converter generating and outputting first and second output elements by converting data types of the first and second fixed point output elements.

First claim

What is claimed is:

1 . A matrix multiplier, comprising:
an input vector scaler configured to generate a first quantization scaled input vector based on a first input vector, on a plurality of common scale coefficients, and on first-to-R-th multiplication scale coefficients, where R is an integer greater than or equal to 2; a first data type converter configured to generate a first fixed point quantization scaled input vector based on the first quantization scaled input vector; a processing element array comprising a first processing element configured to generate a first fixed point output element based on the first fixed point quantization scaled input vector and on a first plurality of quantization sign bits, and a second processing element configured to generate a second fixed point output element based on the first fixed point quantization scaled input vector and on a second plurality of quantization sign bits; and a second data type converter configured to generate first and second output elements by converting data types of the first and second fixed point output elements, respectively, and configured to output a first output vector including the first and second output elements.

2 . The matrix multiplier of claim 1 , wherein,
the first input vector includes first and second input elements, and the first quantization scaled input vector includes: a first plurality of quantization scaled input elements configured to be generated based on the first input element, on a first common scale coefficient corresponding to one of the plurality of common scale coefficients, and on the first-to-Rth multiplication scale coefficients; and a second plurality of quantization scaled input elements configured to be generated based on the second input element, a second common scale coefficient corresponding to one of the plurality of common scale coefficients, and the first-to-Rth multiplication scale coefficients.

3 . The matrix multiplier of claim 2 , wherein the input vector scaler includes:
a multiplication scaling circuit configured to generate first-to-Rth multiplication scaled input elements based on the first input element and the first-to-Rth multiplication scale coefficients, and configured to generate (R+1)th-to-(2R)th multiplication scaled input elements based on the second input element and the first-to-Rth multiplication scale coefficients; and a common scaling circuit configured to generate the first plurality of quantization scaled input elements based on products of the first common scale coefficient with each of the first-to-Rth multiplication scaled input elements, and configured to generate the second plurality of quantization scaled input elements based on products of the second common scale coefficient with each of the (R+1)th-to-(2R)th multiplication scaled input elements.

4 . The matrix multiplier of claim 2 , wherein the input vector scaler includes:
a common scaling circuit configured to generate a first common scaled input element based on a product of the first input element and the first common scale coefficient, and configured to generate a second common scaled input element based on a product of the second input element and the second common scale coefficient; and a multiplication scaling circuit configured to generate the first plurality of quantization scaled input elements based on a product of the first common scaled input element with each of the first-to-Rth multiplication scale coefficients, and configured to generate the second plurality of quantization scaled input elements based on a product of the second common scaled input element with each of the first-to-Rth multiplication scale coefficients.

5 . The matrix multiplier of claim 2 , wherein the input vector scaler includes:
a multiplication scaling circuit configured to generate first-to-Rth quantization scale coefficients based on a product of the first common scale coefficient with each of the first-to-Rth multiplication scale coefficients, and configured to generate (R+1)th-to-(2R)th quantization scale coefficients based on a product of the second common scale coefficient with each of the first-to-Rth multiplication scale coefficients; and a first quantization scaling circuit configured to generate the first plurality of quantization scaled input elements based on the first input element and the first-to-Rth quantization scale coefficients, and configured to generate the second plurality of quantization scaled input elements based on the second input element and the (R+1)th-to-(2R)th quantization scale coefficients.

6 . The matrix multiplier of claim 5 , wherein:
the input vector scaler is further configured to generate a second quantization scaled input vector based on a second input vector, on the plurality of common scale coefficients, and on the first-to-Rth multiplication scale coefficients, the first data type converter is further configured to generate a second fixed point quantization scaled input vector based on the second quantization scaled input vector, the processing element array further comprises a third processing element configured to generate a third fixed point output element based on the second fixed point quantization scaled input vector and on the first plurality of quantization sign bits, and a fourth processing element configured to generate a fourth fixed point output element based on the second fixed point quantization scaled input vector and on the second plurality of quantization sign bits, and the second data type converter is further configured to generate third and fourth output elements by converting data types of the third and fourth fixed point output elements, respectively, and configured to output a second output vector including third and fourth output elements.

7 . The matrix multiplier of claim 6 , wherein:
the second input vector includes third and fourth input elements, and the input vector scaler further includes a second quantization scaling circuit configured to generate a third plurality of quantization scaled input elements included in the second quantization scaled input vector based on the third input element and on the first-to-Rth quantization scale coefficients, and configured to generate a fourth plurality of quantization scaled input elements included in the second quantization scaled input vector based on the fourth Input element and on the (R+1)th-to-(2R)th quantization scale coefficients.

8 . The matrix multiplier of claim 6 , wherein:
the first processing element is arranged in a first processing element row and in a first processing element column of the processing element array, the second processing element is arranged in the first processing element row and in a second processing element column of the processing element array, the third processing element is arranged in a second processing element row and in the first processing element column of the processing element array, and the fourth processing element is arranged in the second processing element row and in the second processing element column of the processing element array.

9 . The matrix multiplier of claim 5 , wherein:
the first common scale coefficient has a floating point data type, and the multiplication scaling circuit is configured to generate the first-to-Rth quantization scale coefficients by change a value of an exponent part of the first common scale coefficient based on the first-to-Rth multiplication scale coefficients.

10 . The matrix multiplier of claim 1 , wherein:
the first-to-Rth multiplication scale coefficients form a geometric sequence with a common ratio of 2.

11 . The matrix multiplier of claim 1 , wherein:
the input v

Assignees

• Samsung Electronics Co Ltd
• Naver Corp

Inventors

• Jeon Younho
• Jang Jae Hun
• Kim Suchang
• Park Yeo-Reum
• Son Hong Rak
• Lim Jihoon
• Kwon Se Jung
• Kim Byeoung Wook
• Park Baeseong
• Lee Dongsoo

Classifications

• G06F17/16 Primary

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06F2207/4824

  Neural networks · CPC title

• G06F7/483

  Computations with numbers represented by a non-linear combination of denominational numbers, e.g. rational numbers, logarithmic number system or floating-point numbers {(G06F7/4806, G06F7/4824, G06F7/49, G06F7/491, G06F7/544 take precedence)} · CPC title

• G06F7/5443 Primary

  Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00) · CPC title