Interoperable neural network operation scheduler

Patent metadata

Field	Value
Publication number	US-10698737-B2
Application number	US-201815963548-A
Country	US
Kind code	B2
Filing date	Apr 26, 2018
Priority date	Apr 26, 2018
Publication date	Jun 30, 2020
Grant date	Jun 30, 2020

Abstract

A Neural Network (NN) scheduler and techniques to implement features of different possible NN schedulers are disclosed. In a first example, an NN scheduler that accepts NN models in an interoperable format and performs optimizations on this interoperable format as part of converting it to a run-time format is provided. In a second example, an NN scheduler analyzes operations and annotations associated with those operations to determine scheduling options based on hardware availability, data availability, hardware efficiency, processor affinity, etc. In a third example, an NN scheduler that may be integrated with a feed-back loop to recognize actual run-time attributes may be used to “learn” and adapt to change its future scheduling behavior. Each of these examples may be integrated individually, or together, to provide an NN scheduler that optimizes and adapts processing functions for an NN model either prior to processing or for just-in-time determination of operation scheduling.

First claim

What is claimed is:

1. A computer-implemented method comprising:
obtaining information describing a first neural network (NN) model in an interoperable data format;
processing the information to determine a plurality of operations representing discrete execution units of the first NN model;
identifying attributes from the information to create descriptor information as annotations describing execution criteria of each operation from the plurality of operations;
obtaining information about hardware functionality and availability for a plurality of hardware processors configured to process the first NN model;
matching a first operation to a first selected hardware processor from the plurality of hardware processors, in part, by comparing the annotations for the first operation with functionality of the plurality of hardware processors;
matching a second operation to a second selected hardware processor from the plurality of hardware processors, in part, by comparing the annotations for the second operation with functionality of the plurality of hardware processors;
creating a first schedule entry and executing the first operation on the first selected hardware processor; and
creating a second schedule entry and executing the second operation on the second selected hardware processor,
wherein hardware performance capabilities of the first selected hardware processor and the second selected hardware processor are different,
at least one of the annotations for the first operation align with capabilities of the first selected hardware processor not available for the second selected hardware processor, and
at least one of the annotations for the second operation align with capabilities of the second selected hardware processor not available for the first selected hardware processor.

2. The computer-implemented method of claim 1 , wherein creating the first and second schedule entries comprises dispatching the first and second schedule entries upon completion of creating schedule entries for the plurality of operations.

3. The computer-implemented method of claim 1 , wherein the first operation is dispatched for execution on the first selected hardware processor prior to creating the second schedule entry for dispatching the second operation.

4. The computer-implemented method of claim 1 , further comprising:
identifying a third operation as having an affinity with the first operation based, in part, on the annotations associated with the first and third operations;
combining the first operation with the third operation to result in a job including both the first operation and the third operation; and
treating the job as the first operation for matching, scheduling, and execution.

5. The computer-implemented method of claim 1 , further comprising:
optimizing the interoperable format in conjunction with processing the information to determine the plurality of operations.

6. The computer-implemented method of claim 1 , further comprising:
determining one or more affinities across a subset of the plurality of operations, the subset representing a group of individual operations;
combining the subset into one or more jobs based on the affinities representing possible efficiencies for execution as determined from the annotations for each individual operation; and
using a composite of the annotations for each individual operation to represent annotations for the one or more jobs for matching, scheduling, and execution, wherein a single job and its corresponding composite annotations represent a single operation with respect to further processing.

7. The computer-implemented method of claim 6 , wherein the one or more affinities represent information indicating each of the group of individual operations requires access to a limited access system resource.

8. The computer-implemented method of claim 7 , wherein the limited access system resource comprises a data resource having limited access across a network of compute resources designated to perform functions for processing the first NN model.

9. The computer-implemented method of claim 1 , further comprising:
converting the interoperable data format into a run-time executable format for a specific compute resource based, in part, on the run-time executable format matching hardware criteria of the specific compute resource.

10. The computer-implemented method of claim 9 , wherein converting comprises cross-compiling to create at least a portion of the run-time executable format.

11. The computer-implemented method of claim 1 , further comprising:
monitoring execution parameters of previously dispatched operations or jobs to collect monitoring information;
updating annotation information based on analysis of the monitoring information to create updated annotation information; and
using the updated annotation information for processing a second NN model.

12. The computer-implemented method of claim 1 , further comprising:
monitoring execution parameters of previously dispatched operations or jobs to collect monitoring information; and
updating scheduling criteria used by a scheduler based on analysis of the monitoring information to adjust future selection of hardware processors.

13. The computer-implemented method of claim 1 , further comprising:
monitoring execution parameters of previously dispatched operations or jobs to collect monitoring information;
updating configuration settings of an adaptive device based on analysis of the monitoring information to create an updated adaptive device; and
using the updated adaptive device for executing at least a portion of a second NN model.

14. A non-transitory computer readable medium comprising computer executable instructions stored thereon that when executed by one or more processing units, perform a method to provide a neural network (NN) operation creation and scheduling function, the method comprising:
obtaining information describing a first neural network (NN) model in an interoperable data format;
processing the information to determine a plurality of operations representing discrete execution units of the first NN model;
identifying attributes from the information to create descriptor information as annotations describing execution criteria of each operation from the plurality of operations;
obtaining information about hardware functionality and availability for a plurality of hardware processors configured to process the first NN model;
matching a first operation to a first selected hardware processor from the plurality of hardware processors, in part, by comparing the annotations for the first operation with functionality of the plurality of hardware processors;
matching a second operation to a second selected hardware processor from the plurality of hardware processors, in part, by comparing the annotations for the second operation with functionality of the plurality of hardware processors;
creating a first schedule entry and executing the first operation on the first selected hardware processor; and
creating a second schedule entry and executing the second operation on the second selected hardware processor,
wherein hardware performance capabilities of the first selected hardware processor and the second selected hardware processor are different,
at least one of the annotations for the first operation align with capabilities of the first selected hardware processor not available for the second selected hardware processor, and
at least one of the annotations for the second operation align with capabilities of the second selected hardware processor not available for the first selected hardware pr

Assignees

• Hewlett Packard Entpr Dev Lp

Inventors

• Fachini Guilherme James De Angelis
• Milojicic Dejan S
• Tomas Gustavo Henrique Rodrigues Pinto
• Silveira Francisco Plinio Oliveira

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/063 Primary

  using electronic means · CPC title

• G06F9/5044

  considering hardware capabilities · CPC title

• G06F9/5055 Primary

  considering software capabilities, i.e. software resources associated or available to the machine · CPC title

• G06N3/105

  Shells for specifying net layout · CPC title