Device and method for generating a compressed network from a trained neural network

Patent metadata

Field	Value
Publication number	US-12024159-B2
Application number	US-202016983802-A
Country	US
Kind code	B2
Filing date	Aug 3, 2020
Priority date	Sep 19, 2019
Publication date	Jul 2, 2024
Grant date	Jul 2, 2024

Abstract

A device and a method for generating a compressed network from a trained neural network are provided. The method includes: a model generating a compressing map from first training data, the compressing map representing the impact of model components of the model to first output data in response to the first training data; generating a compressed network by compressing the trained neural network in accordance with the compressing map; the trained neural network generating trained network output data in response to second training data; the compressed network generating compressed network output data in response to the second training data; training the model by comparing the trained network output data with the compressed network output data.

First claim

What is claimed is:

1. A method of a computer-implemented model generating a compressed network from a trained neural network, the method comprising the following steps:
generating, by a first model portion of the model, an impact map representing an impact of model components for each first output datum of first output data in response to an associated first training datum, wherein each generated impact map includes, for each of the model components, an impact in processing the associated first training datum;
generating a combined impact map using each of the generated impact maps;
generating, by a second model portion of the model, a compressing map from the combined impact map;
generating a compressed network by compressing the trained neural network in accordance with the compressing map, wherein the generating of the compressed network includes deleting network components from the trained neural network in accordance with the compressing map when a corresponding value in the compressing map meets a predefined deleting criterion;
generating, by the trained neural network, trained network output data in response to second training data;
generating, by the compressed network, compressed network output data in response to the second training data; and
training the model by comparing the trained network output data with the compressed network output data.

2. The method of claim 1 , wherein the training of the model includes training the first model portion and/or training the second model portion by comparing the trained network output data with the compressed network output data.

3. The method of claim 1 , wherein the predefined deleting criterion is met when the corresponding value in the compressing map is below a predefined threshold value.

4. The method of claim 1 , wherein the training of the model includes training the model to increase a total compression by reducing a sum of each value of the compressing map.

5. The method of claim 1 , wherein the first output data are generated by the trained neural network for the first training data.

6. The method of claim 1 , wherein the trained neural network is trained to provide first output data for first input data of a plurality of tasks, and wherein the compressed network provides second output data for second input data of at least one task of the plurality of tasks.

7. The method of claim 6 , wherein the first training data and/or the second training data are selected from a plurality of data using a selection model.

8. The method of claim 1 , further comprising the following steps:
generating, by the compressed network generated by the trained model, third training data in response to input data; and
training another model using the third training data.

9. A non-transitory computer-readable memory medium on which is stored a computer program of a computer-implemented model generating a compressed network from a trained neural network, the computer program, when executed by a computer, causing the computer to perform the following steps:
generating, by a first model portion of the model, an impact map representing an impact of model components for each first output datum of first output data in response to an associated first training datum, wherein each generated impact map includes, for each of the model components, an impact in processing the associated first training datum;
generating a combined impact map using each of the generated impact maps;
generating, by a second model portion of the model, a compressing map from the combined impact map;
generating a compressed network by compressing the trained neural network in accordance with the compressing map, wherein the generating of the compressed network includes deleting network components from the trained neural network in accordance with the compressing map when a corresponding value in the compressing map meets a predefined deleting criterion;
generating, by the trained neural network, trained network output data in response to second training data;
generating, by the compressed network, compressed network output data in response to the second training data; and
training the model by comparing the trained network output data with the compressed network output data.

10. A system, comprising:
a device including a compressed network generated by a trained model, the device configured to process digital input data; and
at least one sensor configured to provide digital input data for the device;
wherein the trained model is trained by
generating, by a first model portion of the model, an impact map representing an impact of model components for each first output datum of first output data in response to an associated first training datum, wherein each generated impact map includes, for each of the model components, an impact in processing the associated first training datum;
generating a combined impact map using each of the generated impact maps;
generating, by a second model portion of the model, a compressing map from the combined impact map;
generating a compressed network by compressing a trained neural network in accordance with the compressing map, wherein the generating of the compressed network includes deleting network components from the trained neural network in accordance with the compressing map when a corresponding value in the compressing map meets a predefined deleting criterion;
generating, by the trained neural network, trained network output data in response to second training data;
generating, by the compressed network, compressed network output data in response to the second training data; and
training the model by comparing the trained network output data with the compressed network output data.

11. A vehicle, comprising:
at least one sensor configured to provide digital input data; and
a driving assistance system including a compressed network generated from a trained neural network using a trained model, wherein the trained neural network is configured to process the digital input data, wherein the compressed network is configured to provide digital output data for the digital input data provided by the at least one sensor, and wherein the driving assistance system is configured to control the vehicle using the digital output data, and wherein the trained model is trained by:
generating, by a first model portion of the model, an impact map representing an impact of model components for each first output datum of first output data in response to an associated first training datum, wherein each generated impact map includes, for each of the model components, an impact in processing the associated first training datum;
generating a combined impact map using each of the generated impact maps;
generating, by a second model portion of the model, a compressing map from the combined impact map;
generating a compressed network by compressing the trained neural network in accordance with the compressing map, wherein the generating of the compressed network includes deleting network components from the trained neural network in accordance with the compressing map when a corresponding value in the compressing map meets a predefined deleting criterion;
generating, by the trained neural network, trained network output data in response to second training data;
generating, by the compressed network, compressed network output data in response to the second training data; and
training the model by comparing the trained network output data with the compressed network output data.

Assignees

• Bosch Gmbh Robert

Inventors

• Peters Jorn
• Hoogeboom Emiel
• Welling Max
• Kandemir Melih
• Barsim Karim Said Mahmoud

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/082 Primary

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title

• G06N3/045

  Combinations of networks · CPC title