Dynamic throughput ingestion of backup sources

Patent metadata

Field	Value
Publication number	US-11558457-B2
Application number	US-202117156015-A
Country	US
Kind code	B2
Filing date	Jan 22, 2021
Priority date	Sep 23, 2015
Publication date	Jan 17, 2023
Grant date	Jan 17, 2023

Abstract

A method and apparatus for dynamically adjusting an ingestion rate for backup operations on a source system. The method generally includes monitoring a resource utilization related to one or more performance metrics of the source system in performing at least a primary workload. Based on the monitored resource utilization, the backup system determines a data ingestion rate for backup operations on the source system. The backup system ingests data from the source system to a backup repository at the determined data ingestion rate.

First claim

What is claimed is:

1. A method, comprising:
querying, by an activity monitor of a backup system, a source system to obtain a resource utilization related to one or more performance metrics of the source system in performing at least a primary workload;
adjusting, by a backup engine of the backup system, a data ingestion rate from the source system, wherein the data ingestion rate from the source system is adjusted based on the one or more performance metrics of the source system in performing at least the primary workload, wherein a rate of change at which the data ingestion rate decreases as the primary workload at the source system increases is based on a priority of the primary workload performed by the source system with respect to data ingestion for backup operations; and
ingesting data from the source system at the adjusted data ingestion rate.

2. The method of claim 1 , wherein the data ingestion rate is determined based on the resource utilization.

3. The method of claim 1 , wherein a query of the source system obtains information about read activity, write activity, throughput, storage system latency, and/or processor utilization.

4. The method of claim 1 , further comprising logging instantaneous resource utilization.

5. The method of claim 1 , further comprising logging a moving average of resource utilization.

6. The method of claim 1 , wherein the priority of the primary workload is prioritized over backup operations associated with the backup engine.

7. The method of claim 1 , wherein a priority of backup operations associated with the backup engine is prioritized over the priority of the primary workload.

8. The method of claim 1 , wherein a priority of backup operations associated with the backup engine is equal to the priority of the primary workload.

9. The method of claim 1 , wherein adjusting the data ingesting rate from the source system includes polling for the resource utilization.

10. The method of claim 1 , wherein the data ingestion rate from the source system is adjusted based in part on one or more activity thresholds, wherein the one or more activity thresholds includes a high activity threshold, wherein the high activity threshold indicates a primary workload resource utilization at which the data ingestion rate is to be reduced.

11. The method of claim 10 , further comprising reducing the data ingestion rate from the source system in response to the primary workload resource utilization being greater than the high activity threshold.

12. The method of claim 1 , wherein the data ingestion rate from the source system is adjusted based in part on one or more activity thresholds, wherein the one or more activity thresholds includes a low activity threshold, wherein the low activity threshold indicates a primary workload resource utilization at which the data ingestion rate is to be increased.

13. The method of claim 12 , further comprising increasing the data ingestion rate from the source system in response to the primary workload resource utilization being less than the low activity threshold.

14. The method of claim 1 , wherein the data ingestion rate from the source system is adjusted based on changes to the one or more performance metrics of the source system in performing at least the primary workload.

15. The method of claim 1 , wherein the one or more performance metrics of the source system in performing at least the primary workload include at least one or more of CPU usage, memory usage, or network throughput.

16. The method of claim 1 , wherein the source system is configured to host one or more virtual machines that share resources of the source system.

17. The method of claim 1 , wherein the source system is configured to provide one or more services.

18. The method of claim 1 , wherein the source system is configured to host one or more databases.

19. A non-transitory computer readable storage medium and comprising computer instructions for:
querying, by an activity monitor of a backup system, a source system to obtain a resource utilization related to one or more performance metrics of the source system in performing at least a primary workload;
adjusting, by a backup engine of the backup system, a data ingestion rate from the source system, wherein the data ingestion rate from the source system is adjusted based on the one or more performance metrics of the source system in performing at least the primary workload, wherein a rate of change at which the data ingestion rate decreases as the primary workload at the source system increases is based on a priority of the primary workload performed by the source system with respect to data ingestion for backup operations; and
ingesting data from the source system at the adjusted data ingestion rate.

20. A system, comprising:
a processor configured to:
query, by an activity monitor of a backup system, a source system to obtain a resource utilization related to one or more performance metrics of the source system in performing at least a primary workload;
adjust, by a backup engine of the backup system, a data ingestion rate from the source system, wherein the data ingestion rate from the source system is adjusted based on the one or more performance metrics of the source system in performing at least the primary workload and one or more activity thresholds, wherein a rate of change at which the data ingestion rate decreases as the primary workload at the source system increases is based on a priority of the primary workload performed by the source system with respect to data ingestion for backup operations; and
ingest data from the source system at the adjusted data ingestion rate; and

a memory coupled to the processor and configured to provide the processor with instructions.

Assignees

• Cohesity Inc

Inventors

• Thomas Markose
• Manjunath Chinmaya

Classifications

• H04L43/0888

  Throughput · CPC title

• H04L67/1095 Primary

  Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes · CPC title

• H04L43/0817 Primary

  by checking functioning · CPC title

• H04L67/51

  Discovery or management thereof, e.g. service location protocol [SLP] or web services · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title