Matrix multiplier and operation method of matrix multiplication device including the same

Patent metadata

Field	Value
Publication number	US-2025139194-A1
Application number	US-202418823847-A
Country	US
Kind code	A1
Filing date	Sep 4, 2024
Priority date	Oct 25, 2023
Publication date	May 1, 2025
Grant date	—

Abstract

A matrix multiplier includes an input vector scaler configured to generate a first scaled input vector based on a first input vector and a plurality of quantization scale coefficients, a first data type converter configured to generate a first fixed-point scaled input vector based on the first scaled input vector, a processing element array including a first processing element configured to generate a first fixed-point output element based on the first fixed-point scaled input vector and first plurality of quantization sign values and a second processing element configured to generate a second fixed-point output element based on the first fixed-point scaled input vector and second plurality of quantization sign values, and a second data type converter configured to generate first and second output elements by converting data type of the first and second fixed-point output elements, and to output a first output vector including the first and second output elements.

First claim

1 . A matrix multiplier comprising:
an input vector scaler configured to generate a first scaled input vector based on a first input vector and on a plurality of quantization scale coefficients; a first data type converter configured to generate a first fixed-point scaled input vector based on the first scaled input vector; a processing element array including a first processing element configured to generate a first fixed-point output element based on the first fixed-point scaled input vector and on a first plurality of quantization sign values, and a second processing element configured to generate a second fixed-point output element based on the first fixed-point scaled input vector and on second plurality of quantization sign values; and a second data type converter configured to generate a first output element and a second output element by converting a data type of the first fixed-point output element and a data type of the second fixed-point output element respectively, and configured to output a first output vector including the first and second output elements.

2 . The matrix multiplier of claim 1 , wherein
the input vector scaler is further configured to generate a second scaled input vector based on a second input vector and on the plurality of quantization scale coefficients, the first data type converter is further configured to generate a second fixed-point scaled input vector based on the second scaled input vector, the processing element array further includes a third processing element and a fourth processing element, the third processing element configured to generate a third fixed-point output element based on the second fixed-point scaled input vector and on the first plurality of quantization sign values, and the fourth processing element configured to generate a fourth fixed-point output element based on the second fixed-point scaled input vector and on the second plurality of quantization sign values, and the second data type converter is further configured to generate a third output element and a fourth output element by converting a data type of the third fixed-point output element and a data type of the fourth fixed-point output element respectively, and to output a second output vector including the third and fourth output elements.

3 . The matrix multiplier of claim 2 , wherein the first and second processing elements are in a first processing element row of the processing element array, and the third and fourth processing elements are in a second processing element row of the processing element array.

4 . The matrix multiplier of claim 2 , wherein the first and third processing elements are in a first processing element column of the processing element array, and the second and fourth processing elements are in a second processing element column of the processing element array.

5 . The matrix multiplier of claim 2 , wherein
a dimension of the first fixed-point scaled input vector is R times of a dimension of the first input vector, (wherein R is an integer greater than or equal to 2), and a dimension of the second fixed-point scaled input vector is R times of a dimension of the second input vector.

6 . The matrix multiplier of claim 5 , wherein the dimension of the first input vector, the dimension of the second input vector, the dimension of the first output vector, and the dimension of the second output vector are the same.

7 . The matrix multiplier of claim 2 , wherein the first data type converter comprises:
a first exponent extract circuit configured to extract a first exponent corresponding to a largest one among exponents of each of first plurality of scaled input elements included in the first scaled input vector; a second exponent extract circuit configured to extract a second exponent based on a largest one among exponents of each of second plurality of scaled input elements included in the second scaled input vector; a first data type convert circuit configured to generate the first fixed-point scaled input vector by converting a data type of each of the first plurality of scaled input elements to fixed-point based on the first exponent; and a second data type convert circuit configured to generate the second fixed-point scaled input vector by converting a data type of each of the second plurality of scaled input elements to fixed-point based on the second exponent.

8 . The matrix multiplier of claim 7 , wherein the second data type converter is further configured to:
convert the data types of the first and second fixed-point output elements to floating-point based on the first exponent, and convert the data types of the third and fourth fixed-point output elements to floating-point based on the second exponent.

9 . The matrix multiplier of claim 8 , wherein
exponent parts of the first and second output elements correspond to the first exponent, and exponent parts of the third and fourth output elements correspond to the second exponent.

10 . The matrix multiplier of claim 1 , wherein
the first processing element is configured to generate the first fixed-point output element by accumulating products of first plurality of fixed-point scaled input elements included in the first fixed-point scaled input vector and the first plurality of quantization sign values, and the second processing element is configured to generate the second fixed-point output element by accumulating products of the first plurality of fixed-point scaled input elements included in the second fixed-point scaled input vector and the second plurality of quantization sign values.

11 . The matrix multiplier of claim 10 , wherein the first processing element includes:
an accumulation register; and an arithmetic logic unit (ALU) including a first input terminal configured to sequentially receive the first plurality of fixed-point scaled input elements, a second input terminal configured to sequentially receive the first plurality of quantization sign values, and a third input terminal connected to the accumulation register, and wherein the arithmetic logic unit is configured to update a value stored in the accumulation register with a value obtained by adding a product of one scaled input element received through the first input terminal and one quantization sign value received through the second input terminal to a value received by the third input terminal.

12 . A matrix multiplier comprising:
an input vector scaler configured to generate a first plurality of scaled input elements based on a first input element and a first plurality of quantization scale coefficients, and to generate a second plurality of scaled input elements based on a second input element and a second plurality of quantization scale coefficients; a first data type converter configured to generate a first plurality of fixed-point scaled input elements based on the first plurality of scaled input elements, and to generate a second plurality of fixed-point scaled input elements based on the second plurality of scaled input elements; a first processing element configured to generate a first fixed-point output element by accumulating the first plurality of fixed-point scaled input elements and the second plurality of fixed-point scaled input elements based on a plurality of quantization sign values; and a second data type converter configured to generate a first output element by converting a data type of the first fixed-point output element.

13 . The matrix multiplier of claim 12 , wherein a number of the first plurality of quantization scale coefficients, a number of the second plurality of quantization scale coefficients, a number of the first p

Assignees

• Samsung Electronics Co Ltd
• Naver Corp

Inventors

• Jeon Younho
• Son Hong Rak
• Song Wonsuk
• Yoo Younggeon
• Yoon Jongyoon
• Lim Jihoon
• Jang Jae Hun
• Kwon Sejung
• Kim Byeoungwook
• Park Baeseong
• Lee Dongsoo

Classifications

• G06F17/16 Primary

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06N3/063

  using electronic means · CPC title

• G06F7/485

  Adding; Subtracting {(G06F7/4833, G06F7/4836 take precedence)} · CPC title

• G06F7/57

  Arithmetic logic units [ALU], i.e. arrangements or devices for performing two or more of the operations covered by groups G06F7/483 – G06F7/556 or for performing logical operations {(G06F7/49, G06F7/491 take precedence)} · CPC title

• G06F5/01

  for shifting, e.g. justifying, scaling, normalising {(digital stores in which the information is moved stepwise, e.g. shift-registers G11C19/00; digital stores in which the information circulates G11C21/00)} · CPC title