Isolating resources for utilization by tenants executing in multi-tenant software containers

Patent metadata

Field	Value
Publication number	US-9703611-B1
Application number	US-201414222564-A
Country	US
Kind code	B1
Filing date	Mar 21, 2014
Priority date	Mar 21, 2014
Publication date	Jul 11, 2017
Grant date	Jul 11, 2017

Abstract

Technologies are described herein for isolating tenants executing in a multi-tenant software container. Mechanisms for resource isolation allow tenants executing in a multi-tenant software container to be isolated in order to prevent resource starvation by one or more of the tenants. Mechanisms for dependency isolation may be utilized to prevent one tenant executing in a multi-tenant software container from using another tenant in the same container in a manner that requires co-tenancy. Mechanisms for security isolation may be utilized to prevent one tenant in a multi-tenant software container from accessing protected data or functionality of another tenant. Mechanisms for fault isolation may be utilized to prevent tenants in a multi-tenant software container from causing faults or other types of errors that affect other tenants executing in the same software container.

First claim

What is claimed is:

1. A non-transitory computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by a computer, cause the computer to:
monitor utilization of one or more resources of the computer by a plurality of tenants executing in a multi-tenant software container, the multi-tenant software container executing in a process on a virtual machine hosted by the computer, and the plurality of tenants executing as threads in the process, the monitoring based on one or more of bytecode weaving performed at a virtual machine level, interception performed at an operating system level, or shadow environments;
determine, based upon the monitoring, that at least one tenant of the plurality of tenants is utilizing the one or more resources in excess of a specified threshold, wherein at least one of the resources comprises file descriptors, and wherein the determining comprises:
maintaining a counter for individual ones of the plurality of tenants, the counter indicating a number of file descriptors created by a corresponding tenant, and
determining that the counter for the at least one tenant exceeds a specified file descriptor threshold; and
in response to determining that the at least one tenant is utilizing the one or more resources in excess of the specified threshold, perform one or more actions with regard to the at least one tenant that is utilizing the one or more resources in excess of the specified threshold.

2. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more resources comprise one or more of disk space, CPU cycles, system memory, permanent generation data, young generation data, old generation data, disk I/O operations, disk bandwidth, network I/O operations, network bandwidth, load balancer connections, file descriptors, processes, or execution threads.

3. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more actions comprise one or more of denying a resource request from the at least one tenant, redirecting a resource request from the at least one tenant to another multi-tenant software container, evicting the at least one tenant from the multi-tenant software container, moving the at least one tenant to another multi-tenant software container, reducing CPU cycles allocated to the at least one tenant, pausing execution of the at least one tenant, or throttling execution of the at least one tenant.

4. The non-transitory computer-readable storage medium of claim 1 , wherein the monitoring is further based on one or more of memory dumps or tracing tools.

5. The non-transitory computer-readable storage medium of claim 1 , having further computer-executable instructions stored thereupon which, when executed by the computer, cause the computer to execute the plurality of tenants in respective individual shadow environments in order to monitor the utilization of the one or more resources by individual ones of the plurality of tenants.

6. A system for isolating one or more resources for utilization by a plurality of tenants executing in a multi-tenant software container, the system comprising:
a processor; and
a computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by the processor, cause the system to
monitor utilization of the one or more resources of the system by the plurality of tenants executing in the multi-tenant software container, the multi-tenant software container executing in a process on a virtual machine hosted by the system, and the plurality of tenants executing as threads of the process, the monitoring based on one or more of bytecode weaving performed at a virtual machine level, interception performed at an operating system level, or shadow environments,
determine, based upon the monitoring, that at least one tenant of the plurality of tenants utilizes the one or more resources in excess of a specified threshold, wherein at least one of the resources comprises file descriptors, and wherein the determining comprises:
maintaining a counter for individual ones of the plurality of tenants, the counter indicating a number of file descriptors created by a corresponding tenant, and
determining that the counter for the at least one tenant exceeds a specified file descriptor threshold; and

in response to determining that the at least one tenant utilizes the one or more resources in excess of the specified threshold, perform one or more actions with respect to the at least one tenant of the plurality of tenants that utilizes the one or more resources in excess of the specified threshold.

7. The system of claim 6 , wherein at least one of the resources comprises execution threads, and wherein the computer-readable storage medium has further computer-executable instructions stored thereupon which, when executed by the processor, cause the system to
maintain a counter for individual ones of the plurality of tenants, the counter indicating a number of execution threads created by a corresponding tenant, and
utilize the counter to prevent the corresponding tenant from creating execution threads in excess of a specified execution thread count.

8. The system of claim 7 , wherein at least one counter is generated using one or more of bytecode weaving, interception, or tracing.

9. The system of claim 6 , wherein at least one of the resources comprises execution threads, and wherein the computer-readable storage medium has further computer-executable instructions stored thereupon which, when executed by the processor, cause the system to
maintain data identifying execution threads created by individual ones of the plurality of tenants, and
utilize the data to reap execution threads created by a corresponding tenant if the corresponding tenant is removed from the multi-tenant software container.

10. The system of claim 6 , wherein the computer-readable storage medium has further computer-executable instructions stored thereupon which, when executed by the processor, cause the system to
associate a first directory on a file system with a tenant of the plurality of tenants, and
prevent the tenant from writing to a second directory on the file system that is not associated with the tenant.

11. The system of claim 6 , wherein the computer-readable storage medium has further computer-executable instructions stored thereupon which, when executed by the processor, cause the system to
associate a disk quota with a tenant of the plurality of tenants, and
prevent the tenant from writing an amount of data to a file system in excess of the disk quota associated with the tenant.

12. The system of claim 6 , wherein at least one of the resources comprises memory, and wherein the computer-readable storage medium has further computer-executable instructions stored thereupon which, when executed by the processor, cause the system to
maintain a counter for individual ones of the plurality of tenants, the counter indicating an amount of memory utilized by a corresponding tenant, and
utilize the counter to prevent the corresponding tenant from utilizing memory in excess of a specified memory threshold.

13. The system of claim 12 , wherein at least one counter is generated by bytecode weaving a virtual machine, and wherein the memory comprises one or more of permanent generation data, young generation data, or old generation data.

14. The system of claim 6 , wherein the computer-readable storage medium has further computer-executable instructions stored thereupon which, when executed by the processor, cause the system to execute individual ones of the pluralit

Assignees

• Amazon Tech Inc

Inventors

• Christopher Keian
• Beranek Kevin Michael
• Kaulia Christopher Keakini
• Kulkarni Vijay Ravindra
• Moniz Samuel Leonard
• Peterson Kyle Bradley
• Varangaonkar Ajit Ashok
• Xu Jun

Classifications

• G06F9/5077 Primary

  Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455) · CPC title

• G06F9/5083 Primary

  Techniques for rebalancing the load in a distributed system · CPC title

• G06F9/5072

  Grid computing · CPC title

• G06F11/0712 Primary

  in a virtual computing platform, e.g. logically partitioned systems · CPC title

• G06F9/5027

  the resource being a machine, e.g. CPUs, Servers, Terminals · CPC title