Generating and deploying packages for machine learning at edge devices

Patent metadata

Field	Value
Publication number	US-12293260-B2
Application number	US-201815884279-A
Country	US
Kind code	B2
Filing date	Jan 30, 2018
Priority date	Nov 21, 2017
Publication date	May 6, 2025
Grant date	May 6, 2025

Abstract

A provider network implements a machine learning deployment service for generating and deploying packages to implement machine learning at connected devices. The service may receive from a client an indication of an inference application, a machine learning framework to be used by the inference application, a machine learning model to be used by the inference application, and an edge device to run the inference application. The service may then generate a package based on the inference application, the machine learning framework, the machine learning model, and a hardware platform of the edge device. To generate the package, the service may optimize the model based on the hardware platform of the edge device and/or the machine learning framework. The service may then deploy the package to the edge device. The edge device then installs the inference application and performs actions based on inference data generated by the machine learning model.

First claim

What is claimed is:

1. A system, comprising:
one or more computing devices of a provider network comprising respective processors and memory to implement a machine learning deployment service to:
receive, from a user of a client of the machine learning deployment service via a management interface of the machine learning deployment service a set of indications including:
a first indication, from the user, of an inference application, wherein the inference application comprises one or more functions configured to perform one or more actions based on inference data generated by a machine learning model;
a second indication, from the user, of a machine learning framework to be used by the inference application, wherein the second indication of the machine learning framework is based on a selection by the user of the machine learning framework from among a plurality of machine learning frameworks stored at the provider network that are available for selection, based on user input via the management interface at the provider network, to be used by the inference application when the inference application is executed on at least one connected device of a remote network of the client, wherein the machine learning framework is configured to run at least a portion of a machine learning model;
a third indication, from the user, of the machine learning model to be used by the inference application, wherein the third indication of the machine learning model is based on a selection by the user of the machine learning model from among a plurality of machine learning models stored at the provider network that are available for selection, based on the user input via the management interface at the provider network, to be used by the inference application when the inference application is executed on the at least one connected device of the remote network of the client, wherein the machine learning model is configured to generate the inference data based on collected data; and
a fourth indication, from the user, of at least one connected device of a remote network of the client to run the inference application, wherein the first indication, the second indication, the third indication, and the fourth indication are different indications received from the same user via the management interface;

responsive to receipt, from the user via the management interface of the machine learning deployment service, of the set of indications:
generate a package based at least on the inference application, the machine learning framework, and the machine learning model; and
deploy the package from the provider network to the at least one connected device of the remote network of the client based on the fourth indication, from the user, of the at least one connected device of the remote network of the client to run the inference application,
wherein another package is deployed from the provider network to another connected device to run the inference application using a different machine learning model based at least on an indication of the other connected device and a selection of the different machine learning model from among the plurality of machine learning models.

2. The system as recited in claim 1 , wherein to generate the package, the one or more computing devices are configured to implement the machine learning deployment service to:
determine a hardware platform of the at least one connected device; and
perform modifications to the machine learning model based on the hardware platform of the at least one connected device, wherein the modified machine learning model is optimized for running on the hardware platform.

3. The system as recited in claim 2 , wherein to perform modifications to the machine learning model, the one or more computing devices are further configured to implement the machine learning deployment service to:
perform additional modifications to the machine learning model based on the machine learning framework, wherein the modified machine learning model is optimized for the hardware platform and the machine learning framework.

4. The system as recited in claim 1 , wherein the one or more computing devices are configured to implement the machine learning deployment service to:
receive an indication that an updated version of the machine learning model is available;
retrieve at least the updated machine learning model;
generate another package based at least on the updated machine learning model; and
deploy the other package to the at least one connected device.

5. The system as recited in claim 1 , wherein to generate the package, the one or more computing devices are configured to implement the machine learning deployment service to:
determine a hardware platform of the at least one connected device; and
select, based on the hardware platform of the at least one connected device, a version from among a plurality of versions of the machine learning framework that are pre-configured for different respective hardware platforms, wherein the selected version of the machine learning framework is pre-configured for the hardware platform of the at least one connected device.

6. A method, comprising:
performing, by one or more computing devices of a provider network that implement a machine learning deployment service:
receiving, from a user of a client of the machine learning deployment service, via a management interface of the machine learning deployment service a set of indications including:
a first indication, from the user, of an inference application, wherein the inference application comprises one or more functions configured to perform one or more actions based on inference data generated by a machine learning model;
a second indication, from the user, of a machine learning framework configured to run at least a portion of a machine learning model, wherein the second indication of the machine learning framework is based on a selection by the user of the machine learning framework from among a plurality of machine learning frameworks stored at the provider network that are available for selection, based on user input via the management interface at the provider network, to be used by the inference application when the inference application is executed on at least one connected device of a remote network of the client;
a third indication, from the user, of the machine learning model to be used by the inference application, wherein the third indication of the machine learning model is based on a selection by the user of the machine learning model from among a plurality of machine learning models stored at the provider network that are available for selection, based on the user input via the management interface at the provider network, to be used by the inference application when the inference application is executed on the at least one connected device of the remote network of the client, wherein the machine learning model is configured to generate the inference data; and
a fourth indication, from the user, of at least one connected device of a remote network of the client to run the inference application, wherein the first indication, the second indication, the third indication, and the fourth indication are different indications received from the same user via the management interface; and

responsive to receipt, from the user via the management interface of the machine learning deployment service, of the set of indications:
generating a package based at least on the inference application, the machine learning model, and the machine learning framework; and
deploying the package from the provider network to the at least one connected device of the remote network of the client based on the fourth indication, from the user, of the at least one connected device of the remote network

Assignees

• Amazon Tech Inc

Inventors

• Kuo Calvin Yue-Ren
• Chen Jiazhen
• Sun Jingwei
• Liu Haiyang

Classifications

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F8/60 Primary

  Software deployment · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• H04W4/38

  for collecting sensor information · CPC title

• G06N20/00 Primary

  Machine learning · CPC title