Dynamic throughput ingestion of backup sources

Patent metadata

Field	Value
Publication number	US-10944822-B2
Application number	US-201916373138-A
Country	US
Kind code	B2
Filing date	Apr 2, 2019
Priority date	Sep 23, 2015
Publication date	Mar 9, 2021
Grant date	Mar 9, 2021

Abstract

A method and apparatus for dynamically adjusting an ingestion rate for backup operations on a source system. The method generally includes monitoring a resource utilization related to one or more performance metrics of the source system in performing at least a primary workload. Based on the monitored resource utilization, the backup system determines a data ingestion rate for backup operations on the source system. The backup system ingests data from the source system to a backup repository at the determined data ingestion rate.

First claim

What is claimed is:

1. A method, comprising:
monitoring a resource utilization related to one or more performance metrics of a source system in performing at least a primary workload, wherein a processor is configured to query the source system to obtain the resource utilization related to one or more performance metrics of the source system in performing at least the primary workload;
determining a data ingestion rate for backup operations on the source system based on the monitored resource utilization;
adjusting, by a backup engine of a backup system, the data ingestion rate from the source system, wherein the data ingestion rate from the source system is adjusted based on the one or more performance metrics of the source system in performing at least the primary workload, wherein a rate of change at which the data ingestion rate decreases as the primary workload at the source system increases is based on a priority of the primary workload with respect to data ingestion for backup operations;
ingesting data from the source system at the adjusted data ingestion rate; and
writing the ingested data to a distributed file system at a backup repository.

2. The method of claim 1 , wherein the data ingestion rate from the source system is adjusted based in part on one or more activity thresholds, wherein the one or more activity thresholds includes a high activity threshold, wherein the high activity threshold indicates a primary workload resource utilization at which the data ingestion rate is to be reduced.

3. The method of claim 2 , further comprising reducing the data ingestion rate from the source system in response to the primary workload resource utilization being greater than the high activity threshold.

4. The method of claim 1 , wherein the data ingestion rate from the source system is adjusted based in part on one or more activity thresholds, wherein the one or more activity thresholds includes a low activity threshold, wherein the low activity threshold indicates a primary workload resource utilization at which the data ingestion rate is to be increased.

5. The method of claim 4 , further comprising increasing the data ingestion rate from the source system in response to the primary workload resource utilization being less than the low activity threshold.

6. The method of claim 1 , further comprising determining whether the backup operations are completed.

7. The method of claim 6 , further comprising adjusting the data ingestion rate from the source system based on changes to the one or more performance metrics of the source system in performing at least the primary workload.

8. The method of claim 1 , wherein the one or more performance metrics of the source system in performing at least the primary workload include at least one or more of CPU usage, memory usage, or network throughput.

9. The method of claim 1 , wherein the source system is configured to host one or more virtual machines that share resources of the source system.

10. The method of claim 1 , wherein the source system is configured to provide one or more services.

11. The method of claim 1 , wherein the source system is configured to be an internet server.

12. The method of claim 1 , wherein the source system is configured to host one or more databases.

13. The method of claim 1 , wherein the source system is configured to provide a test environment for software updates.

14. The method of claim 1 , wherein the source system is configured to be a production system that is deploying a current version of software.

15. A non-transitory computer readable storage medium and comprising computer instructions for:
monitoring a resource utilization related to one or more performance metrics of a source system in performing at least a primary workload, wherein a processor is configured to query the source system to obtain the resource utilization related to one or more performance metrics of the source system in performing at least the primary workload;
determining a data ingestion rate for backup operations on the source system based on the monitored resource utilization;
adjusting, by a backup engine of a backup system, the data ingestion rate from the source system, wherein the data ingestion rate from the source system is adjusted based on the one or more performance metrics of the source system in performing at least the primary workload, wherein a rate of change at which the data ingestion rate decreases as the primary workload at the source system increases is based on a priority of the primary workload with respect to data ingestion for backup operations;
ingesting data from the source system at the adjusted data ingestion rate; and
writing the ingested data to a distributed file system at a storage repository.

16. The non-transitory computer readable storage medium of claim 15 , wherein the data ingestion rate from the source system is adjusted based in part on one or more activity thresholds, wherein the one or more activity thresholds includes a high activity threshold, wherein the high activity threshold indicates a primary workload resource utilization at which the data ingestion rate is to be reduced.

17. The non-transitory computer readable storage medium of claim 16 , further comprising instructions for reducing the data ingestion rate from the source system in response to the primary workload resource utilization being greater than the high activity threshold.

18. The non-transitory computer readable storage medium of claim 15 , wherein the data ingestion rate from the source system is adjusted based in part on one or more activity thresholds, wherein the one or more activity thresholds includes a low activity threshold, wherein the low activity threshold indicates a primary workload resource utilization at which the data ingestion rate is to be increased.

19. The non-transitory computer readable storage medium of claim 18 , further comprising instructions for increasing the data ingestion rate from the source system in response to the primary workload resource utilization being less than the low activity threshold.

20. A system, comprising:
a processor configured to:
monitor a resource utilization related to one or more performance metrics of a source system in performing at least a primary workload, wherein the processor is configured to query the source system to obtain the resource utilization related to one or more performance metrics of the source system in performing at least the primary workload;
determine a data ingestion rate for backup operations on the source system based on the monitored resource utilization;
adjust, by a backup engine of a backup system, the data ingestion rate from the source system, wherein the data ingestion rate from the source system is adjusted based on the one or more performance metrics of the source system in performing at least the primary workload and one or more activity thresholds, wherein a rate of change at which the data ingestion rate decreases as the primary workload at the source system increases is based on a priority of the primary workload with respect to data ingestion for backup operations;
ingest data from the source system at the adjusted data ingestion rate; and
write the ingested data to a distributed file system at a storage repository; and

a memory coupled to the processor and configured to provide the processor with instructions.

Assignees

• Cohesity Inc

Inventors

• Thomas Markose
• Manjunath Chinmaya

Classifications

• H04L43/0817 Primary

  by checking functioning · CPC title

• H04L67/51

  Discovery or management thereof, e.g. service location protocol [SLP] or web services · CPC title

• H04L43/16

  Threshold monitoring · CPC title

• H04L67/1095 Primary

  Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes · CPC title

• H04L43/04

  Processing captured monitoring data, e.g. for logfile generation · CPC title