Dynamic resource scaling for large-scale backups

Patent metadata

Field	Value
Publication number	US-12229012-B2
Application number	US-202217989493-A
Country	US
Kind code	B2
Filing date	Nov 17, 2022
Priority date	Sep 27, 2022
Publication date	Feb 18, 2025
Grant date	Feb 18, 2025

Abstract

Methods, systems, and devices for data management are described. A data management system (DMS) may determine to generate a backup instance of an application that includes multiple user accounts. The DMS may identify a time window for generation of the backup instance based on a set of backup parameters associated with the application. The DMS may calculate respective runtime durations for the user accounts based on respective quantities of data objects in the user accounts and respective backup job types associated with the data objects. The DMS may determine respective values of a semaphore parameter based on the respective runtime durations and the time window, where the semaphore parameter defines a threshold quantity of data objects to be processed in parallel. The DMS may generate the backup instance of the application based on processing the user accounts in accordance with the respective values of the semaphore parameter.

First claim

What is claimed is:

1. A method for data management, comprising:
determining, by a data management system, to generate a backup instance of an application that comprises a plurality of user accounts;
identifying, by the data management system, a time window during which the data management system will generate the backup instance based at least in part on a set of backup parameters associated with the application;
calculating, by the data management system respective runtime durations associated with generation of the backup instance for the plurality of user accounts, the respective runtime durations based at least in part on respective quantities of data objects in the plurality of user accounts and respective backup job types associated with the data objects;
determining, by the data management system, for the plurality of user accounts, respective values of a semaphore parameter based at least in part on the respective runtime durations and the time window, wherein the semaphore parameter corresponds to a threshold quantity of backup processes to be performed in parallel as part of the generation of the backup instance, wherein performing a backup process comprises backing up a respective data object; and
generating, by the data management system during the time window, the backup instance of the application based at least in part on processing the plurality of user accounts in accordance with the respective values of the semaphore parameter.

2. The method of claim 1 , wherein generating the backup instance of the application comprises:
executing, by the data management system, one or more full backup jobs to generate full backups for a first set of one or more data objects in a user account of the plurality of user accounts; and
executing, by the data management system, one or more incremental backup jobs to generate incremental backups for a second set of one or more data objects in the user account.

3. The method of claim 2 , wherein the plurality of user accounts are associated with a client of the data management system, and wherein calculating the respective runtime durations for the plurality of user accounts comprises:
calculating, by the data management system, a runtime duration for the one or more full backup jobs based at least in part on an average runtime duration of previous full backup jobs associated with the client of the data management system.

4. The method of claim 2 , wherein calculating the respective runtime durations for the plurality of user accounts comprises:
calculating, by the data management system, a runtime duration for the one or more incremental backup jobs based at least in part on an average runtime duration of previous incremental backup jobs associated with the application.

5. The method of claim 2 , wherein calculating the respective runtime durations for the plurality of user accounts comprises:
calculating, by the data management system, a first runtime duration for the one or more full backup jobs and a second runtime duration for the one or more full backup jobs, wherein a runtime duration for the user account is based at least in part on a combination of the first runtime duration and the second runtime duration.

6. The method of claim 1 , wherein determining the respective values for the semaphore parameter comprises:
determining, by the data management system, a value of the semaphore parameter for a user account that supports all data objects in the user account being processed within the time window.

7. The method of claim 1 , further comprising:
updating, by the data management system, the respective values of the semaphore parameter according to a computational load of the data management system, a processing capacity of the data management system, a completion status of the backup instance, or any combination thereof.

8. The method of claim 1 , wherein generating the backup instance of the application comprises:
executing, by the data management system, one or more backup jobs to generate backups of the data objects in the plurality of user accounts; and
storing, by the data management system, runtime information associated with the one or more backup jobs in association with the backup instance of the application.

9. The method of claim 1 , wherein generating the backup instance of the application comprises:
executing, by the data management system, an incremental backup job to generate an incremental backup of a data object in a user account of the plurality of user accounts; and
executing, by the data management system, a full backup job to generate a full backup of the data object in response to detecting a failure associated with the incremental backup job.

10. The method of claim 9 , further comprising:
storing, by the data management system, information associated with the failure, wherein the respective values of the semaphore parameter are calculated based at least in part on the stored information associated with the failure.

11. The method of claim 9 , wherein the failure comprises an application programming interface (API) failure, a network failure, a dependent service failure, or any combination thereof.

12. The method of claim 1 , wherein generating the backup instance of the application comprises:
executing, by the data management system, an incremental backup job to generate an incremental backup of a data object in a user account of the plurality of user accounts, wherein executing the incremental backup job comprises:
transmitting, to a data service that supports the application, an application programming interface (API) request indicating a key associated with a prior full backup of the data object; and
receiving, from the data service, an API response indicating one or more changes made to the data object between a first time associated with generation of the prior full backup and a second time associated with transmission of the API request, wherein the incremental backup of the data object comprises the one or more changes indicated by the API response.

13. The method of claim 1 , wherein determining the respective values of the semaphore parameter comprises:
determining, by the data management system, a buffer percentage for the semaphore parameter based at least in part on a failure tolerance of the data management system.

14. The method of claim 13 , further comprising:
increasing, by the data management system, a value of the semaphore parameter for a user account in response to detecting a backup job failure associated with the user account, wherein the value of the semaphore parameter for the user account is increased by a percentage that is less than or equal to the buffer percentage.

15. The method of claim 1 , wherein the respective values of the semaphore parameter are determined in accordance with a schedule that is based at least in part on a compliance status of the application.

16. The method of claim 1 , wherein determining the respective values of the semaphore parameter comprises:
reducing, by the data management system, a value of the semaphore parameter for a user account of the plurality of user accounts after generating full backups of all data objects in the user account.

17. The method of claim 1 , wherein the data objects comprise emails, files, documents, account settings, contacts, or any combination thereof.

18. The method of claim 1 , wherein the set of backup parameters indicate the time window, a full backup policy for the application, an incremental backup policy for the application, a retention policy for the backup

Assignees

• Rubrik Inc

Inventors

• AGRAWAL Rohit
• Pandey Prateek
• Terei David
• Thulasingam Ramya

Classifications

• G06F2201/84

  Using snapshots, i.e. a logical point-in-time copy of the data · CPC title

• G06F11/1451 Primary

  by selection of backup contents · CPC title

• G06F11/1461 Primary

  Backup scheduling policy · CPC title