Distributed resource management by improving cluster diversity

Patent metadata

Field	Value
Publication number	US-11716384-B2
Application number	US-202117302585-A
Country	US
Kind code	B2
Filing date	May 6, 2021
Priority date	Nov 20, 2018
Publication date	Aug 1, 2023
Grant date	Aug 1, 2023

Abstract

A method of distributed resource management in a distributed computing system includes determining usage of respective hardware resources by an application and generating usage metrics for the application, and assigning the application to a cluster of hardware resources to optimize diversity of usage of hardware resources in the cluster and to enhance utilization of the hardware resources by applications running in that cluster. The diversity of usage of the hardware resources is determined from respective usage metrics of the respective applications running in that cluster. The diversity of usage of the hardware resources in the cluster is optimized by assigning the application to a diversity pool of hardware resources adapted to minimize interference when applications assigned to the diversity pool of hardware resources access the hardware resources in the diversity pool and assigning applications from different diversity pools to the cluster of hardware resources.

First claim

What is claimed is:

1. A computer-implemented method of distributed resource management in a distributed computing system, comprising:
determining usage of low-level hardware resources by an application and generating usage metrics for the application;
identifying a preferred subset of the low-level hardware resources in a cluster of hardware resources, the preferred subset identified based on low-level hardware diversity that reduces interference among the low-level hardware resources;
determining hardware characteristics of a plurality of hardware resource nodes within the cluster of hardware resources;
generating a plurality of hardware resource node diversity scores based on the hardware characteristics, each of the plurality of hardware resource node diversity scores indicating a hardware node low-level resource diversity;
selecting a least diverse hardware resource node from among the plurality of hardware resource nodes based on the plurality of hardware resource node diversity scores, the least diverse hardware resource node associated with a lowest number of different low-level hardware resource types; and
assigning the application to the least diverse hardware resource node to optimize a diversity of usage of the preferred subset of the low-level hardware resources in the cluster of hardware resources and to enhance utilization of hardware resources by applications running in the cluster of hardware resources, the diversity of usage of the low-level hardware resources in the cluster of hardware resources being determined from usage metrics of the applications running in the cluster of hardware resources.

2. The method of claim 1 , wherein the determining of the usage of the low-level hardware resources by the application comprises using a hardware performance counter to determine at least one of a cache coherence, a memory bandwidth, a multi-level cache performance, a processor bus performance, a processor pipeline performance, a cache bandwidth, a cache hit rate, or non-uniform memory access latencies, the hardware performance counter including a set of special-purpose registers built into a microprocessor.

3. The method of claim 1 , wherein the determining of the usage of low-level hardware resources by the application to determine the usage metrics is performed when the application is running offline.

4. The method of claim 1 , wherein the determining of the usage of hardware resources by the application to determine the usage metrics is performed in real time as the application is being executed by the low-level hardware resources.

5. The method of claim 1 , further comprising characterizing the low-level hardware resources using micro-benchmarking to identify performance-critical hardware resources.

6. The method of claim 1 , wherein the assigning of the application to the cluster of hardware resources to optimize the diversity of usage of the low-level hardware resources in the cluster of hardware resources comprises:
extrapolating hardware resources required by the application to a capability of each hardware resource; and
assigning the application to a diversity pool of hardware resources based on a fairness algorithm to maximize the diversity of usage of the low-level hardware resources in the diversity pool based on the usage metrics.

7. The method of claim 6 , wherein the assigning of the application to the cluster of hardware resources to optimize the diversity of usage of the low-level hardware resources in the cluster of hardware resources comprises sorting the application and other applications into diversity pools of hardware resources using the fairness algorithm.

8. The method of claim 1 , wherein the assigning of the application to the cluster of hardware resources to optimize the diversity of usage of the low-level hardware resources in the cluster of hardware resources comprises:
assigning the application to a diversity pool of hardware resources adapted to minimize interference when applications assigned to the diversity pool of hardware resources access the low-level hardware resources in the diversity pool; and
assigning applications from different diversity pools to the cluster of hardware resources.

9. The method of claim 1 , wherein the assigning of the application to the cluster of hardware resources to optimize the diversity of usage of the low-level hardware resources in the cluster of hardware resources comprises:
selecting a least diverse cluster of hardware resources; and
assigning applications from respective diversity pools to the least diverse cluster of hardware resources for processing.

10. An apparatus for providing distributed resource management in a distributed computing system, comprising:
at least one hardware performance counter that generates usage metrics for low-level hardware resources used by an application;
a memory storing instructions; and
at least one processor in communication with the memory, the at least one processor configured, upon execution of the instructions, to act as a resource manager and perform the following steps:
identifies a preferred subset of the low-level hardware resources in a cluster of hardware resources, the preferred subset identified based on low-level hardware diversity that reduces interference among the low-level hardware resources;
determines hardware characteristics of a plurality of hardware resource nodes within the cluster of hardware resources;
generates a plurality of hardware resource node diversity scores based on the hardware characteristics, each of the plurality of hardware resource node diversity scores indicating a hardware node low-level resource diversity;
selects a least diverse hardware resource node from among the plurality of hardware resource nodes based on the plurality of hardware resource node diversity scores, the least diverse hardware resource node associated with a lowest number of different low-level hardware resource types; and
assigns the application to the least diverse hardware resource node based on the usage metrics to optimize a diversity of usage of the preferred subset of the low-level hardware resources in the cluster of hardware resources and to enhance utilization of the low-level hardware resources by applications running in the cluster of hardware resources, the diversity of usage of the low-level hardware resources in the cluster of hardware resources being determined from usage metrics of the applications running in the cluster of hardware resources.

11. The apparatus of claim 10 , wherein the at least one hardware performance counter determines the usage metrics for the application when the application is running offline.

12. The apparatus of claim 10 , wherein the at least one hardware performance counter determines the usage metrics for the application in real time as the application is being executed by the low-level hardware resources.

13. The apparatus of claim 10 , further comprising a micro-benchmarking application that is run on a node including the low-level hardware resources to identify performance critical hardware resources of the node.

14. The apparatus of claim 10 , wherein the at least one manager further executes the instructions to:
assigns the application to the cluster of hardware resources to optimize the diversity of usage of the low-level hardware resources in the cluster of hardware resources by extrapolating hardware resources required by the application to a capability of each hardware resource; and
assigns the application to a diversity pool of hardware resources based on a fairness algorithm to maximize the diversity of usage of the low-level hardware reso

Assignees

• Huawei Tech Co Ltd

Inventors

• Srikanthan Sharanyan
• Lin Zongfang
• Tian Chen
• Hu Ziang

Classifications

• H04L67/1097 Primary

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• H04L67/34

  involving the movement of software or configuration parameters  (network booting or remote initial program loading [RIPL] G06F9/4416) · CPC title

• H04L67/535

  Tracking the activity of the user (network monitoring arrangements H04L43/00; recording of computer activity G06F11/34) · CPC title

• G06F9/5027 Primary

  the resource being a machine, e.g. CPUs, Servers, Terminals · CPC title

• G06F2209/508

  Monitor · CPC title