Pre-emptive container load-balancing, auto-scaling and placement

Patent metadata

Field	Value
Publication number	US-11604682-B2
Application number	US-202017139210-A
Country	US
Kind code	B2
Filing date	Dec 31, 2020
Priority date	Dec 31, 2020
Publication date	Mar 14, 2023
Grant date	Mar 14, 2023

Abstract

A resource usage platform is disclosed. The platform performs preemptive container load balancing, auto scaling, and placement in a computing system. Resource usage data is collected from containers and used to train a model that generates inferences regarding resource usage. The resource usage operations are performed based on the inferences and on environment data such as available resources, service needs, and hardware requirements.

First claim

What is claimed is:

1. A method, comprising:
receiving a request to invoke a container in a computing system from a container image;
generating an inference from an inference service by providing an input to the inference service, wherein the inference includes a prediction of resources needed for the request;
receiving environment data associated with the computing system, the environment data including information about resources available at nodes in the computing system;
attaching services information to the container that describe services used by the container;
scheduling the container to run on a node in the computing system based on the inference and the environment data and based on the services information attached to the container, wherein the node has access to the services; and
running the container on the node.

2. The method of claim 1 , wherein the environment data comprises container metadata including hardware requirements, resource usage data, and service requirements.

3. The method of claim 1 , further comprising training a model with telemetry data collected from running containers in the computing system.

4. The method of claim 3 , further comprising collecting telemetry data from each container on each node in the computing system.

5. The method of claim 1 , wherein the environment data includes resources consumed by each container at each node, resources available at each node, and execution times of each container.

6. The method of claim 1 , further comprising scheduling the request in an existing and running container, scheduling the request in a new container on a selected node, or rejecting the request.

7. The method of claim 1 , further comprising training a model using telemetry data and/or aggregating the telemetry data.

8. The method of claim 1 , further comprising auto scaling containers in the computing system and/or performing federating learning.

9. A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
receiving a request to invoke a container in a computing system from a container image;
generating an inference from an inference service by providing an input to the inference service, wherein the inference includes a prediction of resources needed for the request;
receiving environment data associated with the computing system, the environment data including information about resources available at nodes in the computing system;
attaching services information to the container that describe services used by the container;
scheduling the container to run on a node in the computing system based on the inference and the environment data and based on the services information attached to the container, wherein the node has access to the services; and
running the container on the node.

10. The non-transitory storage medium of claim 9 , wherein the environment data comprises container metadata including hardware requirements, resource usage data, and service requirements.

11. The non-transitory storage medium of claim 9 , further comprising training a model with telemetry data collected from running containers in the computing system.

12. The non-transitory storage medium of claim 11 , further comprising collecting telemetry data from each container on each node in the computing system.

13. The non-transitory storage medium of claim 9 , wherein the environment data includes resources consumed by each container at each node, resources available at each node, and execution times of each container.

14. The non-transitory storage medium of claim 9 , further comprising scheduling the request in an existing and running container, scheduling the request in a new container on a selected node, or rejecting the request.

15. The non-transitory storage medium of claim 9 , further comprising training a model using telemetry data and/or aggregating the telemetry data.

16. The non-transitory storage medium of claim 9 , further comprising auto scaling containers in the computing system and/or performing federating learning.

Assignees

• Emc Ip Holding Co Llc

Inventors

• He Xuebin
• Seibel Amy N
• Arora Himanshu
• Fong Victor

Classifications

• G06N3/098

  Distributed learning, e.g. federated learning · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/08

  Learning methods · CPC title

• G06F9/5083 Primary

  Techniques for rebalancing the load in a distributed system · CPC title

• G06F2209/5019

  Workload prediction · CPC title