Inference model selection method considering task request rate

Patent metadata

Field	Value
Publication number	US-2025094840-A1
Application number	US-202418443445-A
Country	US
Kind code	A1
Filing date	Feb 16, 2024
Priority date	Sep 14, 2023
Publication date	Mar 20, 2025
Grant date	—

Abstract

Disclosed is an inference model selection method and apparatus considering a task request rate. The inference model selection method is performed by a computing device including a processor and includes monitoring computing resources of the computing device; receiving a task; selecting an inference model to perform inference for the received task; inputting the task into a queue of the selected inference model; and performing an inference operation.

First claim

What is claimed is:

1 . A n inference model selection method performed by a computing device comprising a processor, the interference model selection method comprising:
monitoring computing resources of the computing device; receiving a task; selecting an inference model to perform inference for the received task; inputting the task into a queue of the selected inference model; and performing an inference operation.

2 . The inference model selection method of claim 1 , wherein the monitoring is periodically performed at predetermined periods.

3 . The inference model selection method of claim 2 , wherein the selecting of the inference model comprises selecting an appropriate inference model for tasks received over the predetermined period, and
the inference model includes at least one Single-task Learning (STL) model and at least one Multi-task Learning (MTL) model.

4 . The inference model selection method of claim 3 , wherein the selecting of the inference model comprises solving a Q-function derived as a result of transforming a Markov Decision Process (MDP) model to Q-Learning, and
the Q-function is represented as Q,(S,A)=max π [Q π (S,A)], where S denotes a state space, A denotes an action space, and π denotes a policy.

5 . The inference model selection method of claim 4 , wherein the performing of the inference operation comprises performing the inference operation by inputting a task input to a queue to the STL model without a waiting time if the STL model is selected.

6 . The inference model selection method of claim 5 , wherein the performing of the inference operation comprises performing the inference operation by, when the queue is full of tasks, inputting the task input to the queue to the MTL model if the MTL model is selected.

7 . The inference model selection method of claim 5 , wherein the performing of the inference operation comprises performing the inference operation by, when a sum of a time elapsed from a point in time at which at least one task stored in the queue is received and a computing time required for inference is equal to a value acquired by subtracting a predetermined period of time from a required delay time, inputting all the tasks stored in the queue to the MTL model, if the MTL model is selected.

8 . The inference model selection method of claim 5 , wherein the performing of the inference operation comprises performing the inference operation by, when a sum of a waiting time of a task in the queue and a computing time required for inference is equal to a value acquired by subtracting a predetermined period of time from a required delay time, inputting all the tasks stored in the queue to the MTL model, if the MTL model is selected.

9 . The inference model selection method of claim 8 , further comprising:
transmitting an inference result.

Assignees

• Univ Korea Res & Bus Found

Inventors

• Pack Sangheon
• Ko Haneul
• Jeong Hyeonjae

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N5/04 Primary

  Inference or reasoning models · CPC title

• G06N3/006

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title