Dynamic split computing framework in serverless edge computing

Patent metadata

Field	Value
Publication number	US-12155535-B2
Application number	US-202318110083-A
Country	US
Kind code	B2
Filing date	Feb 15, 2023
Priority date	Oct 17, 2022
Publication date	Nov 26, 2024
Grant date	Nov 26, 2024

Abstract

Disclosed is a split computing device operating in a serverless edge computing environment. The split computing device includes a transceiver configured to receive resource information of a terminal from the terminal and to measure a data transmission rate between the terminal and the split computing device in a process of receiving the resource information of the terminal; and a splitting point deriver configured to determine a splitting point of a deep neural network (DNN) model for split computing and an activation status of a container instance for each of tail models of a DNN corresponding to the respective splitting points using the resource information of the terminal, the data transmission rate, and resource information of the split computing device.

First claim

What is claimed is:

1. A split computing device operating in a serverless edge computing environment, the split computing device comprising:
a transceiver configured to receive resource information of a terminal from the terminal and to measure a data transmission rate between the terminal and the split computing device in a process of receiving the resource information from the terminal;
a splitting point deriver configured to determine a splitting point of a deep neural network (DNN) model and an activation status of a container instance for each of tail models of the DNN for split computing using the resource information of the terminal, the data transmission rate, and resource information of the split computing device,
wherein the transceiver is configured to receive, from the terminal, intermediate data that is inference results for a head model of the DNN model for the determined splitting point; and
an inference unit configured to derive final results of the DNN model by performing inference on the intermediate data to derive inference results for a tail model of the DNN model for the determined splitting point,
wherein the inference results derived by the inference unit are transmitted to the terminal via the transceiver.

2. The split computing device of claim 1 , wherein the splitting point of the DNN model determined by the splitting point deriver is transmitted to the terminal via the transceiver.

3. The split computing device of claim 1 , wherein the received resource information of the terminal and the intermediate data are stored in a storage.

4. The split computing device of claim 1 , wherein the resource information of the terminal includes available computing power of the terminal and the resource information of the split computing device includes available computing power of the split computing device, and
wherein the splitting point deriver is configured to determine the splitting point capable of minimizing an inference latency while maintaining a computing power of the terminal below a first threshold and a computing power of the split computing device below a second threshold.

5. The split computing device of claim 4 , wherein the inference latency is a sum of an inference latency of a head model for each splitting point, a transmission latency due to transmission of intermediate data corresponding to results of the head model, and an inference latency of a tail model.

Assignees

• Univ Korea Res & Bus Found

Inventors

• Pack Sangheon
• Ko Haneul
• Jung Daeyoung
• Jeong Hyeon Jae

Classifications

• H04L43/0894 Primary

  Packet rate · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/045 Primary

  Combinations of networks · CPC title

• Y02D10/00

  Energy efficient computing, e.g. low power processors, power management or thermal management · CPC title