Dynamic horizontal autoscaling for multi-tenant service

Patent metadata

Field	Value
Publication number	US-12341861-B2
Application number	US-202218066339-A
Country	US
Kind code	B2
Filing date	Dec 15, 2022
Priority date	Dec 15, 2022
Publication date	Jun 24, 2025
Grant date	Jun 24, 2025

Abstract

Embodiments relate to dynamic horizontal autoscaling for a multi-tenant service. According to an aspect, a computer-implemented method includes configuring a network computing element to route a plurality of requests from two or more clients to a service in a containerized-networked system. A horizontal service-level autoscaler monitors one or more aspects of the requests to the service from the two or more clients. A new pod including a set of running containers with shared resources in the containerized-networked system is created based at least in part on the horizontal service-level autoscaler determining that at least one of the one or more aspects of the requests exceeds a threshold. A new service mapping is created to accept subsequent requests from one of the two or more clients that triggered creation of the new pod.

First claim

What is claimed is:

1. A computer-implemented method comprising:
configuring a network computing element to route a plurality of requests from two or more clients to a service in a containerized-networked system, wherein the two or more clients use the same service address to request the service through the network computing element, and the network computing element uses routing rules to route each of the client requests to a particular client-specific variation of a plurality of client-specific variations of the service each having a different service name that represents the service;
monitoring, by a horizontal service-level autoscaler, one or more aspects of the requests to the service from the two or more clients that are received through the network computing element and routed by the network computing element to a first variation of the service using a first service name of the service, wherein the first variation of the service is mapped to a first pod that is a shared pod of all of the two or more clients;
creating a new pod comprising a set of running containers with shared resources in the containerized-networked system, the creating based at least in part on the horizontal service-level autoscaler determining that at least one of the one or more aspects of the requests exceeds a threshold;
creating a new service mapping comprising a second variation of the service having a second service name that maps to the first pod and to the new pod, wherein the new pod is a dedicated pod of the client that triggered creation of the new pod, wherein the new pod may be converted from the dedicated pod to a partially shared pod that is shared by more than one of the clients and shared by fewer than all of the clients; and
updating routing rules in the network computing element by mapping the client that triggered creation of the new pod to the second service name instead of the first service name, thereby causing the network computing element to route subsequent requests from the client that triggered creation of the new pod to the second variation of the service instead of the first variation of the service.

2. The computer-implemented method of claim 1 , wherein the one or more aspects of the requests are checked with respect to a plurality of predefined rules based at least in part on one or more of memory usage and processing resource usage.

3. The computer-implemented method of claim 1 , further comprising: using a label selector to bind a relationship between the new service mapping and the new pod.

4. The computer-implemented method of claim 1 , further comprising: continuing to perform horizontal scaling at a service level for each new client that accesses the network computing element.

5. A system comprising:
a memory; and
a processing device coupled to the memory, the processing device configured to execute instructions to:
configure a network computing element to route a plurality of requests from two or more clients to a service in a containerized-networked system, wherein the two or more clients use the same service address to request the service through the network computing element, and the network computing element uses routing rules to route each of the client requests to a particular client-specific variation of a plurality of client-specific variations of the service each having a different service name that represents the service;
monitor, by a horizontal service-level autoscaler, one or more aspects of the requests to the service from the two or more clients that are received through the network computing element and routed by the network computing element to a first variation of the service using a first service name of the service, wherein the first variation of the service is mapped to a first pod that is a shared pod of all of the two or more clients;
create a new pod comprising a set of running containers with shared resources in the containerized-networked system, the creating based at least in part on the horizontal service-level autoscaler determining that at least one of the one or more aspects of the requests exceeds a threshold;
create a new service mapping comprising a second variation of the service having a second service name that maps to the first pod and to the new pod, wherein the new pod is a dedicated pod of the client that triggered creation of the new pod, wherein the new pod may be converted from the dedicated pod to a partially shared pod that is shared by more than one of the clients and shared by fewer than all of the clients; and
update routing rules in the network computing element by mapping the client that triggered creation of the new pod to the second service name instead of the first service name, thereby causing the network computing element to route subsequent requests from the client that triggered creation of the new pod to the second variation of the service instead of the first variation of the service.

6. The system of claim 5 , wherein the one or more aspects of the requests are checked with respect to a plurality of predefined rules based at least in part on one or more of memory usage and processing resource usage.

7. The system of claim 5 , wherein the processing device and the memory are further configured to use a label selector to bind a relationship between the new service mapping and the new pod.

8. The system of claim 5 , wherein the processing device and the memory are further configured to continue to perform horizontal scaling at a service level for each new client that accesses the network computing element.

9. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more processors to cause the one or more processors to perform operations comprising:
configuring a network computing element to route a plurality of requests from two or more clients to a service in a containerized-networked system, wherein the two or more clients use the same service address to request the service through the network computing element, and the network computing element uses routing rules to route each of the client requests to a particular client-specific variation of a plurality of client-specific variations of the service each having a different service name that represents the service;
monitoring, by a horizontal service-level autoscaler, one or more aspects of the requests to the service from the two or more clients that are received through the network computing element and routed by the network computing element to a first variation of the service using a first service name of the service, wherein the first variation of the service is mapped to a first pod that is a shared pod of all of the two or more clients;
creating a new pod comprising a set of running containers with shared resources in the containerized-networked system, the creating based at least in part on the horizontal service-level autoscaler determining that at least one of the one or more aspects of the requests exceeds a threshold;
creating a new service mapping comprising a second variation of the service having a second service name that maps to the first pod and to the new pod, wherein the new pod is a dedicated pod of the client that triggered creation of the new pod, wherein the new pod may be converted from the dedicated pod to a partially shared pod that is shared by more than one of the clients and shared by fewer than all of the clients; and
updating routing rules in the network computing element by mapping the client that triggered creation of the new pod to the second service name instead of the first service name, thereby causing the network computing element to route subsequent requests from

Assignees

• IBM

Inventors

• Fu Rong
• Yin Jian Dong
• Sheng Hao
• Yu Chuan Qing
• Wang Wen
• Zhang Kang

Classifications

• H04L67/61

  taking into account QoS or priority requirements · CPC title

• H04L43/16

  Threshold monitoring · CPC title

• H04L67/63 Primary

  Routing a service request depending on the request content or context · CPC title