Estimation of resources utilized by deep learning applications

Patent metadata

Field	Value
Publication number	US-12393835-B2
Application number	US-201916381650-A
Country	US
Kind code	B2
Filing date	Apr 11, 2019
Priority date	Apr 20, 2018
Publication date	Aug 19, 2025
Grant date	Aug 19, 2025

Abstract

According to an embodiment, a performance benchmark database is obtained, where the performance benchmark database at least includes structural data of one or more deep neural network models, time performance data and computing resource consumption data of a plurality of deep learning applications based on the one or more deep neural network models; a training dataset is extracted based on the performance benchmark database, where the training dataset has a plurality of parameter dimensions, the plurality of parameter dimensions including: structures of deep neural network models of the plurality of deep learning applications, resource configuration of the plurality of deep learning applications, and training time of the plurality of deep learning applications; and correspondence among the parameter dimensions of the training dataset is created so as to create an estimation model for estimating resources utilized by deep learning applications.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
obtaining a performance benchmark database, wherein the performance benchmark database includes at least structural data of one or more deep neural network models, time performance data comprising at least an average training time of a plurality of deep learning applications based on the one or more deep neural network models, computing resource consumption data comprising at least one of power consumption data, memory consumption data, and storage consumption data of the plurality of deep learning applications based on the one or more deep neural network models, and computing resource configuration data of the plurality of deep learning applications comprising at least a processing unit associated with the one or more deep neural network models of the plurality of deep learning applications;
extracting a training dataset based on the performance benchmark database, wherein the training dataset has a plurality of parameter dimensions, the plurality of parameter dimensions include at least portions of the structural data, the time performance data, the computing resource consumption data, and the computing resource configuration data of the one or more deep neural network models of the plurality of deep learning applications;
creating an estimation model for estimating resources utilized by the plurality of deep learning applications, wherein creating the estimation model comprises creating correspondence between ones of the plurality of parameter dimensions of the extracted training dataset to meet a given criteria, and wherein the given criteria comprises maintaining a linear relation between the plurality of parameter dimensions of the structural data of the one or more deep neural network models of the plurality of deep learning applications and a plurality of other parameter dimensions, and wherein the estimation model is a machine learning model;
allocating system available resources based on the estimation model; and
causing the allocated system available resources to execute one or more given deep learning applications;
wherein the one or more deep neural network models of the plurality of deep learning applications comprise one or more convolutional neural network models
wherein the structural data of the one or more deep neural network models of the plurality of deep learning applications comprises:
structural parameters associated with a computing strength of a convolutional layer of a convolutional neural network model of the one or more convolutional neural network models, wherein the computing strength of the convolutional layer is based on a number of filters of the convolutional layer; and
structural parameters associated with a computing strength of a fully-connected layer of a convolutional neural network model of the one or more convolutional neural network models, wherein the computing strength of the fully-connected layer is based on a number of nodes of the fully-connected layer;

wherein the number of filters of a given convolutional layer and an output of a previous convolutional layer determine the number of parameters of the given convolutional layer to meet the given criteria; and
wherein the maintaining a linear relation between the plurality of parameter dimensions of structures of deep neural network models of the plurality of deep learning applications and the plurality of other parameter dimensions comprises maintaining a linear relation between a number of trainable parameters and changes of the number of filters at the given convolutional layer to meet the given criteria.

2. The method according to claim 1 , wherein the obtaining the performance benchmark database comprises:
running, under different running conditions, at least one sample workload program having a customized deep neural network model, wherein the different running conditions are defined by at least different value combinations of: a computing resource configuration and input data size; and
obtaining training time of the at least one sample workload program having the customized deep neural network model under the different running conditions,
wherein the structural data of the one or more deep neural network models in the performance benchmark database includes structural data of the customized deep neural network model, and the time performance data in the performance benchmark database includes the obtained training time.

3. The method according to claim 2 , wherein the obtaining the training time of the at least one sample workload program having a customized deep neural network model under the different running conditions comprises:
running the at least one sample workload program having a customized deep neural network model at least once under a same running condition;
obtaining a training time of a first N+1 steps of a sample workload program; and
calculating an average training time from step 2 to step N+1.

4. The method according to claim 1 , wherein the plurality of parameter dimensions of the training dataset further comprises one or more of:
an input dataset size;
a resource utilization rate; and
hyper-parameters for deep learning applications.

5. The method according to claim 1 , wherein creating the estimation model for estimating resources utilized by deep learning applications is based on minimizing resource consumption.

6. A computer program product being tangibly stored on a non-transitory computer readable medium and comprising machine executable instructions, which, when executed, cause the machine to perform a method according to claim 1 .

7. A device, comprising:
a processing unit; and
a memory coupled to the processing unit and containing instructions stored thereon, which, when executed by the processing unit, cause the device to perform acts of:
obtaining a performance benchmark database, wherein the performance benchmark database includes at least structural data of one or more deep neural network models, time performance data comprising at least an average training time of a plurality of deep learning applications based on the one or more deep neural network models, computing resource consumption data comprising at least one of power consumption data, memory consumption data, and storage consumption data of the plurality of deep learning applications based on the one or more deep neural network models, and computing resource configuration data of the plurality of deep learning applications comprising at least a processing unit associated with the one or more deep neural network models of the plurality of deep learning applications;
extracting a training dataset based on the performance benchmark database, wherein the training dataset has a plurality of parameter dimensions, the plurality of parameter dimensions include at least portions of the structural data, the time performance data, the computing resource consumption data, and the computing resource configuration data of the one or more deep neural network models of the plurality of deep learning applications;
creating an estimation model for estimating resources utilized by the plurality of deep learning applications, wherein creating the estimation model comprises creating correspondence between ones of the plurality of parameter dimensions of the extracted training dataset to meet a given criteria, and wherein the given criteria comprises maintaining a linear relation between the plurality of parameter dimensions of the structural data of the one or more deep neural network models of the plurality of deep learning applications and a plurality of other parameter dimensions, and wherein the estimation model is a machine learning model;
allocating system available resources based on the estimation model; and
causi

Assignees

• Emc Ip Holding Co Llc

Inventors

• Li Sanping
• Wang Kun

Classifications

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• Y02D10/00

  Energy efficient computing, e.g. low power processors, power management or thermal management · CPC title

• G06N3/045

  Combinations of networks · CPC title