Adaptation of deep learning models to resource constrained edge devices

Patent metadata

Field	Value
Publication number	US-11928583-B2
Application number	US-201916504353-A
Country	US
Kind code	B2
Filing date	Jul 8, 2019
Priority date	Jul 8, 2019
Publication date	Mar 12, 2024
Grant date	Mar 12, 2024

Abstract

Techniques for generating a set of Deep Learning (DL) models are described. An example method includes training an initial set of DL models using the training data, wherein a topology of each of the DL models is determined based on the parameters vector. The method also includes generating a set of estimate performance functions for each of the DL models in the initial set based on the set of edge-related metrics, and generating a plurality of objective functions based on the set of estimated performance functions. The method also includes generating a final DL model set based on the objective functions, receiving a user selection of a selected DL model from the final DL model set, and deploying the selected DL model to an edge device.

First claim

What is claimed is:

1. A system, comprising:
a storage device to store a training corpus comprising training data, a parameters vector, and a plurality of edge-related metrics that are related to a plurality of resource constraints associated with an edge device; and
a processor to:
train an initial set of Deep Learning (DL) models comprising different topologies using the training data, wherein a topology of each of the initial set of DL models is determined based on the parameters vector;
wherein, in training of the initial set of DL models, the processor is to generate a set of estimated performance functions for each of the DL models in the initial set for each of the plurality of edge-related metrics, wherein each of the estimated performance functions comprises a set of values that are computed for each of the plurality of edge-related metrics that describe how a performance of each of the DL models changes throughout a range of parameters with respect to each of the plurality of edge-related metrics;
for each of the plurality of edge-related metrics, generate a plurality of objective functions based on the generated set of estimated performance functions;
train a plurality of final DL models selected from the initial set of models based on a multi objective optimization of the generated plurality of objective functions, wherein the plurality of final DL models are trained using new model parameters and a new estimated performance function is computed for each of the plurality of edge-related metrics with respect to each of the plurality of final DL models, wherein the new estimated performance functions are used to generate an updated plurality of objective functions and produce a new set of final DL models in response to detecting that a difference between the new estimated performance function of a final DL model from the plurality of final DL models, and the estimated performance functions of a DL model from the initial set of DL models, exceeds a threshold error criterion;
receive a user selection of a selected DL model from the new set of final DL models; and
deploy the selected DL model on the edge device.

2. The system of claim 1 , wherein the processor is to adjust a topology of one of the models in the final DL model set upon a determination that the performance of the DL model predicted by the objective functions differs from the actual performance of the DL model by the threshold error criterion.

3. The system of claim 1 , wherein to receive the user selection comprises to generate a user interface that enables a user to specify an objective and displays a ranked list of top ranked DL models ranked in accordance with the specified objective.

4. The system of claim 3 , wherein the processor is to deploy a plurality of DL models to the edge device, and wherein each of the DL models make predictions based on common DL model input, with a final prediction to be determined based on a voting scheme.

5. The system of claim 1 , wherein to generate the final DL model set based on the objective functions comprises to compute a Pareto front of a plot of DL model parameters versus DL performance as computed by the objective functions.

6. The system of claim 1 , wherein the plurality of edge-related metrics comprise an inference time, a model size, and a test accuracy.

7. The system of claim 1 , wherein the parameters vector comprises values describing a number of layers and a number of nodes per layer for each model in the initial set of DL models.

8. The system of claim 1 , wherein the selected DL model is a classifier.

9. The system of claim 1 , wherein the multi-objective optimization comprises conflicting objectives corresponding to different optimal solutions.

10. A computer-implemented method, the computer-implemented method comprising:
training an initial set of Deep Learning (DL) models comprising different topologies on training data, wherein a topology of each of the initial set of DL models is determined based on a parameters vector;
wherein training the initial set of DL models comprises generating a set of estimate performance functions for each of the DL models in the initial set for each of a plurality of edge-related metrics that are related to a plurality of resource constraints associated with an edge device, wherein each of the estimated performance functions comprises a set of values that are computed for each of the plurality of edge-related metrics that describe how a performance of each of the DL models changes throughout a range of parameters with respect to each of the plurality of edge-related metrics;
for each of the plurality of edge-related metrics, generating a plurality of objective functions based on the generated set of estimated performance functions;
training a plurality of final DL models selected from the initial set of models based on a multi objective optimization of the generated plurality of objective functions, wherein the plurality of final DL models are trained using new model parameters and a new estimated performance function is computed for each of the plurality of edge-related metrics with respect to each of the plurality of final DL models, wherein the new estimated performance functions are used to generate an updated plurality of objective functions and produce a new set of final DL models in response to detecting that a difference between the new estimated performance function of a final DL model from the plurality of final DL models, and the estimated performance functions of a DL model from the initial set of DL models, exceeds a threshold error criterion;
receiving a user selection of selected DL model from the new set of final DL models; and
deploying the selected DL model on the edge device.

11. The computer-implemented method of claim 10 , comprising adjusting a topology of one of the models in the final DL model set upon a determination that the performance of the DL model predicted by the objective functions differs from the actual performance of the DL model by a threshold error criterion.

12. The computer-implemented method of claim 10 , wherein receiving the user selection comprises generating a user interface that enables a user to specify an objective and displaying, at the user interface, a ranked list of top ranked DL models ranked in accordance with the specified objective.

13. The computer-implemented method of claim 10 , comprising deploying a plurality of DL models to the edge device, and wherein each of the DL models make predictions based on common DL model input, with a final prediction to be determined based on a voting scheme.

14. The computer-implemented method of claim 10 , wherein generating the final DL model set based on the objective functions comprises to compute a Pareto front of a plot of DL model parameters versus DL performance as computed by the objective functions.

15. The computer-implemented method of claim 10 , wherein the plurality of edge-related metrics comprise an inference time, a model size, and a test accuracy.

16. The computer-implemented method of claim 10 , wherein the parameters vector comprises values describing a number of layers and a number of nodes per layer for each model in the initial set of DL models.

17. The computer-implemented method of claim 10 , wherein the selected DL model is a classifier.

18. A computer program product for deploying Deep Learning (DL) models on edge devices comprising a computer readable storage medium having program instructions embodied therewith, and wherein the program instructions are executable by a processor t

Assignees

• IBM

Inventors

• Turgeman Lior
• Naaman Nir
• Masin Michael
• Guy Nili
• Kalner Shmuel
• Rosen Ira
• Amir Adar

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/098 Primary

  Distributed learning, e.g. federated learning · CPC title

• G06N3/0985

  Hyperparameter optimisation; Meta-learning; Learning-to-learn · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/082

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title