Automatic tuning of artificial neural networks

Patent metadata

Field	Value
Publication number	US-11423311-B2
Application number	US-201615154650-A
Country	US
Kind code	B2
Filing date	May 13, 2016
Priority date	Jun 4, 2015
Publication date	Aug 23, 2022
Grant date	Aug 23, 2022

Abstract

Tuning a neural network may include selecting a portion of a first neural network for modification to increase computational efficiency and generating, using a processor, a second neural network based upon the first neural network by modifying the selected portion of the first neural network while offline.

First claim

What is claimed is:

1. A method of tuning a neural network, the method comprising:
selecting a portion of a first neural network for modification to alter one or more target performance requirements;
generating, using a processor, a second neural network based upon the first neural network by modifying the selected portion of the first neural network while executing neither the first neural network nor the second neural network; and
validating, by the processor, that an operation of the second neural network achieves at least a selected set of the target performance requirements.

2. The method of claim 1 , further comprising:
receiving data indicative of the target performance requirement(s);
wherein the data indicative of the target performance requirement(s) includes an indication of a preference between a first target performance requirement and a second target performance requirement; and
wherein modifying of the selected portion of the first neural network includes altering the selected portion such that the first target performance requirement is more successfully met than the second target performance requirement.

3. The method of claim 1 , further comprising:
receiving data indicative of the first neural network; and
in response to receiving a request to modify the first neural network, automatically performing the selecting of the portion of the first neural network and the generating of the second neural network.

4. The method of claim 1 , further comprising:
retraining the second neural network.

5. The method of claim 1 , wherein the operation of the second neural network further comprises, if the operation of the second neural network does not achieve the selected set of the target performance requirements,
iteratively generating at least one new version of the second neural network, based upon additional modifications to the selected portion of the first neural network.

6. The method of claim 1 , wherein the selected portion of the first neural network comprises weights and the modifying the selected portion of the first neural network comprises:
adjusting selected ones of the weights of the selected portion of the first neural network.

7. The method of claim 6 , wherein adjusting selected ones of the weights of the selected portion of the first neural network comprises:
replacing a convolution kernel of a layer of the first neural network with a replacement convolution kernel.

8. The method of claim 6 , wherein adjusting selected ones of the weights of the selected portion of the first neural network comprises:
scaling a convolution kernel of the first neural network.

9. The method of claim 1 , wherein the modifying the selected portion of the first neural network includes performing an operation selected from the group consisting of convolution kernel substitution, pruning, decomposition, and scaling.

10. The method of claim 1 , wherein the modifying the selected portion of the first neural network comprises:
pruning the portion of the first neural network.

11. The method of claim 10 , wherein pruning the portion of the first neural network comprises performing an operation selected from the group consisting of using a different numerical format for weights of the first neural network, removing a feature map of a layer of the first neural network, zeroing a convolution kernel of the first neural network, and removing a layer of the first neural network.

12. The method of claim 1 , wherein the modifying the selected portion of the first neural network comprises:
replacing an activation function of a neuron of the selected portion of the first neural network with a different activation function.

13. The method of claim 1 , wherein the modifying the selected portion of the first neural network comprises:
decomposing a convolution kernel of the selected portion of the first neural network.

14. The method of claim 1 , wherein the modifying the selected portion of the first neural network comprises:
performing kernel fusion.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Brothers John W
• Lee Joohoon

Classifications

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/082 Primary

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title

• G06N3/045 Primary

  Combinations of networks · CPC title