Intelligent management of machine learning inference in edge-cloud systems

Patent metadata

Field	Value
Publication number	US-12580824-B2
Application number	US-202318541996-A
Country	US
Kind code	B2
Filing date	Dec 15, 2023
Priority date	Dec 15, 2023
Publication date	Mar 17, 2026
Grant date	Mar 17, 2026

Abstract

A system and method relate to managing a cloud computing system. Queries are received from one or more edge devices of a set of edge devices. Each query includes sensor data from the respective edge device. Prediction data is generated via one or more cloud machine learning models, using the sensor data, during a current time period. System state data is generated and indicates a current state of an environment during the current time period. The environment is defined by the cloud computing system and the set of edge devices. A machine learning system generates policy data by optimizing an expected return of a reward with respect to taking a particular action given the system state data. The machine learning system is employed by the cloud computing system. The policy data indicates a recommended action from a set of actions. The cloud computing system performs the recommended action.

First claim

The invention claimed is:

1 . A computer-implemented method for managing a cloud computing system, the computer-implemented method comprising:
receiving queries from one or more edge devices of a set of edge devices that are connected to the cloud computing system, each query including sensor data from the respective edge device; generating prediction data, via one or more machine learning models, using the sensor data, the one or more machine learning models being employed by the cloud computing system during a current time period; generating system state data indicating a current state of an environment during the current time period, the environment being defined by the cloud computing system and the set of edge devices; generating, via a reinforcement learning (RL) agent, policy data by optimizing an expected return of a reward with respect to taking a particular action given the system state data, the policy data indicating a recommended action from a set of actions; and performing the recommended action, wherein the cloud computing system employs the RL agent as a machine learning system to control an offload tendency of transmitting a respective query of a machine learning inference task from one or more of the edge devices to the cloud computing system via the policy data.

2 . The computer-implemented method of claim 1 , wherein:
the machine learning system comprises a deep neural network (DNN) that is configured to optimize the expected return by approximating a Q-value for each action of the set of actions; and the recommended action returns a particular action having the best Q-value from among the set of actions.

3 . The computer-implemented method of claim 1 , wherein the system state data is generated for the current time period based on
a number of the edge devices being serviced by the cloud computing system; a change in the number of edge devices between a previous time period and the current time period, an amount of computer resources that are being used to process the queries; a number of the queries being processed by the cloud computing system; latency data regarding the queries being processed by the cloud computing system; cost data of the cloud computing system incurred by processing the queries from the previous time period to the current time period; and query threshold data being used by each edge device to determine whether or not to generate a respective query for transmission to the cloud computing system.

4 . The computer-implemented method of claim 1 , wherein the set of actions comprises
allocating one or more computer resources of the cloud computing system, deallocating the one or more computer resources, maintaining the one or more computer resources, and modifying a query threshold of the cloud computing system.

5 . The computer-implemented method of claim 1 , further comprising:
generating query threshold data of the cloud computing system for the current time period, the query threshold data being used by each edge device to determine whether or not to generate a respective query for transmission to the cloud computing system; and transmitting the query threshold data to each edge device of the set of edge devices.

6 . The computer-implemented method of claim 1 , wherein the reward is computed such that a predetermined cost budget is not exceeded by cost data associated with processing the queries on the cloud computing system and additional cost data associated with performing the particular action.

7 . The computer-implemented method of claim 1 , wherein the reward is computed such that a predetermined latency target is not exceeded by latency data associated with processing the queries.

8 . A system comprising:
one or more processors; and one or more memory in data communication with the one or more processors, the one or more memory including computer readable data stored thereon that, when executed by the one or more processors, causes the one or more processors to perform a method for managing a cloud computing system, the method including receiving queries from one or more edge devices of a set of edge devices that are connected to the cloud computing system, each query including sensor data from the respective edge device; generating prediction data, via one or more machine learning models, using the sensor data, the one or more machine learning models being employed by the cloud computing system during a current time period; generating system state data indicating a current state of an environment during the current time period, the environment being defined by the cloud computing system and the set of edge devices; generating, via a reinforcement learning (RL) agent, policy data by optimizing an expected return of a reward with respect to taking a particular action given the system state data, the policy data indicating a recommended action from a set of actions; and performing the recommended action, wherein the cloud computing system employs the RL agent as a machine learning system to control an offload tendency of transmitting a respective query of a machine learning inference task from one or more of the edge devices to the cloud computing system via the policy data.

9 . The system of claim 8 , wherein:
the machine learning system comprises a deep neural network (DNN) that is configured to optimize the expected return by approximating a Q-value for each action of the set of actions; and the recommended action returns a particular action having the best Q-value from among the set of actions.

10 . The system of claim 8 , wherein the system state data is generated for the current time period based on
a number of the edge devices being serviced by the cloud computing system; a change in the number of edge devices between a previous time period and the current time period, an amount of computer resources that are being used to process the queries; a number of the queries being processed by the cloud computing system; latency data regarding the queries being processed by the cloud computing system; cost data of the cloud computing system incurred by processing the queries from the previous time period to the current time period; and query threshold data being used by each edge device to determine whether or not to generate a respective query for transmission to the cloud computing system.

11 . The system of claim 8 , wherein the set of actions comprises
allocating one or more computer resources of the cloud computing system, deallocating the one or more computer resources, maintaining the one or more computer resources, and modifying a query threshold of the cloud computing system.

12 . The system of claim 8 , wherein the method further comprises:
generating query threshold data of the cloud computing system for the current time period, the query threshold data being used by each edge device to determine whether or not to generate a respective query for transmission to the cloud computing system; and transmitting the query threshold data to each edge device of the set of edge devices.

13 . The system of claim 8 , wherein the reward is computed such that a predetermined cost budget is not exceeded by cost data associated with processing the queries on the cloud computing system and additional cost data associated with performing the particular action.

14 . The system of claim 8 , wherein the reward is computed such that a predetermined latency target is not exceeded by latency data associated with processing the queries.

15 . One or more non-transitory computer-readable media

Assignees

• Bosch Gmbh Robert

Inventors

• Gopal Sharath
• Li Baolin
• Gualtieri Marcus
• Zhou Xiaowei
• Ren Liu

Classifications

• H04L41/0826 Primary

  for reduction of network costs (H04L41/0833 takes precedence) · CPC title

• H04L41/16 Primary

  using machine learning or artificial intelligence · CPC title

• G06F16/903

  Querying (for retrieval from the web G06F16/953) · CPC title

• G06N20/00

  Machine learning · CPC title

• G06N5/041 Primary

  Abduction · CPC title