Neural network method and apparatus

Patent metadata

Field	Value
Publication number	US-2020005146-A1
Application number	US-201916564494-A
Country	US
Kind code	A1
Filing date	Sep 9, 2019
Priority date	Jul 28, 2016
Publication date	Jan 2, 2020
Grant date	—

Abstract

A lightened neural network, method, and apparatus, and recognition method and apparatus implementing the same. A neural network includes a plurality of layers each comprising neurons and plural synapses connecting neurons included in neighboring layers. Synaptic weights with values greater than zero and less than a preset value of a variable a, which is greater than zero, may be at least partially set to zero. Synaptic weights with values greater than a preset value of a variable b, which is greater than zero, may be at least partially set to the preset value of the variable b.

First claim

What is claimed is:

1 . A neural network apparatus comprising:
a processor configured to obtain and implement a neural network with a plurality of layers each comprising neurons, with plural synapses connecting neurons included in neighboring layers of the plurality of layers, wherein the plural synapses include synapses having synaptic weights, as obtained in the obtaining of the neural network, for interpreting data input to a layer of the plurality of layers having the plural synapses for a trained objective, wherein the synaptic weights include weights, previously having pre-regularized values greater than zero and less than a value of a variable a that is greater than zero, that are regularized values having been set to zero, wherein the synaptic weights include weights, previously having pre-regularized values greater than a value of a variable b that is greater than the value of the variable a, that are additional regularized values having been set to the value of the variable b.

2 . The neural network apparatus of claim 1 , wherein the value of the variable b is different between at least two of the plurality of layers.

3 . The neural network apparatus of claim 1 ,
for the obtaining of the synaptic weights, the processor is further configured to change, based on a predetermined shift increment corresponding to the layer, values of a plurality of compressed weights of the layer acquired from a memory, and for the implementing of the neural network, the processor is configured to implement the layer using, as plural values of the synaptic weights, at least the changed values of the plurality of the compressed weights.

4 . The neural network apparatus of claim 3 , wherein the predetermined shift increment is dependent on the value of the variable a.

5 . The neural network apparatus of claim 3 ,
wherein the processor is further configured to obtain a regularization variable that indicates the predetermined shift increment corresponding to the layer, and wherein the changing of the values of the plurality of compressed weights is an increasing, based on the obtained regularization variable, of the values of the plurality of compressed weights.

6 . The neural network apparatus of claim 3 , wherein the changing of the values of the plurality of compressed weights includes changing, based on a same shift increment, values for each of compressed weights corresponding to the regularized values, compressed weights corresponding to the additional regularized values, and compressed weights corresponding to the weights that are dispersed values.

7 . The neural network apparatus of claim 6 , wherein the changing is an increasing, based on the same shift increment, of the values for each of the compressed weights corresponding to the regularized values, the compressed weights corresponding to the additional regularized values, and the compressed weights corresponding to the weights that are the dispersed values.

8 . The neural network apparatus of claim 6 , wherein the same shift increment is independently set for each of two or more of the plurality of layers or for two or more of a plurality of output map channels included in the layer.

9 . The neural network apparatus of claim 6 ,
wherein the obtaining of the neural network further includes an obtaining of information relating to the variable a, and wherein the same shift increment is performed based on the obtained information related to the variable a.

10 . The neural network apparatus of claim 6 , wherein each of the plurality of compressed weights of the layer acquired from the memory that have values less than or equal a difference value, corresponding to the value of the variable b minus the value of the variable a, is represented by a total number of bits corresponding to log 2(b−a) in which the variable b and the variable a are integers.

11 . The neural network apparatus of claim 1 , wherein each of the synaptic weights with values less than or equal to the value of the variable b is represented by a total number of bits corresponding to log 2(b) in which the variable b is an integer.

12 . A neural network apparatus comprising:
a processor configured to: obtain synaptic weights for synapses of a neural network, where the neural network includes a plurality of layers that each include plural neurons, with plural synapses connecting neurons included in neighboring layers of the plurality of layers; and implement, using the synaptic weights, at least a layer of the neural network without weight sharing, wherein the synaptic weights include weights, previously having pre-regularized values greater than zero and less than a value of a variable a that is greater than zero, that are regularized values having been set to zero, wherein the synaptic weights include weights, previously having pre-regularized values greater than a value of a variable b that is greater than the value of the variable a, that are additional regularized values having been set to the value of the variable b.

13 . The neural network apparatus of claim 12 , wherein the value of the variable b is different between at least two of the plurality of layers.

14 . The neural network apparatus of claim 12 , wherein each of the synaptic weights with values less than or equal to the value of the variable b is represented by a total number of bits corresponding to log 2(b) in which the variable b is an integer.

15 . The neural network apparatus of claim 12 ,
for the obtaining of the synaptic weights, the processor is further configured to change, based on a predetermined shift increment corresponding to the layer, values of a plurality of compressed weights of the layer acquired from a memory, and for the implementing of the neural network, the processor is configured to implement the layer using at least the changed values of the plurality of compressed weights, as the obtained synaptic weights.

16 . The neural network apparatus of claim 15 , wherein the predetermined shift increment is dependent on the variable a.

17 . The neural network apparatus of claim 15 , wherein each of the plurality of compressed weights of the layer acquired from the memory that have values less than or equal a difference value, corresponding to the value of the variable b minus the value of the variable a, is represented by a total number of bits corresponding to log 2(b−a) in which the variable b and the variable a are integers.

18 . The neural network apparatus of claim 15 , wherein the changing of the values of the plurality of compressed weights includes increasing, based on a determined same shift increment, values for each of compressed weights corresponding to the regularized values, compressed weights corresponding to the additional regularized values, and compressed weights corresponding to weights that are values dispersed, between the value of the variable b and the value of the variable a, based on previous training of the layer.

19 . A neural network apparatus comprising:
a processor configured to: obtain synaptic weights for synapses for a layer of a neural network, where the layer of the neural network includes plural neurons, and where the synaptic weights correspond to weighted connections for respective inputs to the plural neurons; and implement the layer of the neural network with the synaptic weights, wherein the synaptic weights include weights, previously having pre-regularized values greater than zero and less than a value of a variable a that is greater than zero, that ar

Assignees

• Samsung Electronics Co Ltd

Inventors

• Son Changyong
• Son Jinwoo
• Yoo Byungin
• Choi Chang Kyu
• Han Jae-Joon

Classifications

• G06N3/082

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title

• G06F7/523

  Multiplying only · CPC title

• G06N3/063 Primary

  using electronic means · CPC title

• G06N3/04 Primary

  Architecture, e.g. interconnection topology · CPC title

• G06F5/01

  for shifting, e.g. justifying, scaling, normalising {(digital stores in which the information is moved stepwise, e.g. shift-registers G11C19/00; digital stores in which the information circulates G11C21/00)} · CPC title