Transaction log index generation in an enterprise backup system

Patent metadata

Field	Value
Publication number	US-11263173-B2
Application number	US-201916526699-A
Country	US
Kind code	B2
Filing date	Jul 30, 2019
Priority date	Jul 30, 2019
Publication date	Mar 1, 2022
Grant date	Mar 1, 2022

Abstract

Certain embodiments disclosed herein reduce or eliminate a communication bottleneck at the storage manager by reducing communication with the storage manager while maintaining functionality of an information management system. In some implementations, operations performed as part of a backup process may be stored in transaction logs. These transaction logs may include information about a transaction performed between the client computing system and the network storage that hosts the backup of the client computing system. The transaction logs may be provided to a secondary storage system that can be used to form a backup index. The backup index may be used to facilitate accessing the data stored at the network storage. Advantageously, generating the transaction logs and separating the generation of the backup index from the backup process can reduce resource usage during performance of the backup and speed up the backup process while further reducing interaction with the storage manager.

First claim

What is claimed is:

1. A computer-implemented method of reducing storage manager overhead associated with secondary storage management of an information management system during backup, the computer-implemented method comprising:
as implemented by one or more hardware processors of the information management system, the one or more hardware processors configured with specific computer-executable instructions,
accessing first backup metadata associated with a first backup of data from a first client computing system of the information management system to one or more secondary storage devices of a network storage system as part of a backup process, the first backup performed using a cached job identifier associated with the first client computing system, wherein the network storage system is external to the information management system;
generating a first transaction log file based on the first backup metadata;
providing the first transaction log file to a first media agent of the information management system associated with the first client computing system;
maintaining the first transaction log file at a plurality of media agents by at least replicating the first transaction log file to at least a second media agent of the information management system, wherein the second media agent is selected from the plurality of media agents based at least in part on an identity of the first client computing system or a job identifier associated with the first transaction log file;
determining occurrence of a trigger to update a backup index at the network storage system;
accessing a set of transaction log files including the first transaction log file;
updating the backup index at the network storage system based on the set of transaction log files; and
flushing the set of transaction log files.

2. The computer-implemented method of claim 1 , further comprising, prior to said flushing the set of transaction log files, replicating the backup index within a distributed index storage system.

3. The computer-implemented method of claim 2 , wherein the distributed index storage system is implemented at the network storage system.

4. The computer-implemented method of claim 2 , wherein updating the backup index comprises identifying a backup index at the distributed index storage system associated with the first client computing system based on the job identifier assigned to the first client computing system.

5. The computer-implemented method of claim 1 , wherein flushing the set of transaction log files comprises deleting the set of transaction log files.

6. The computer-implemented method of claim 1 , wherein the trigger to update the backup index comprises one or more of: an elapse of a period of time, an indication of completion of a backup job, the number of transaction log files added to the set of transaction log files satisfying a threshold; or receipt of a command to update the backup index.

7. The computer-implemented method of claim 1 , wherein updating the backup index comprises generating a new backup index.

8. The computer-implemented method of claim 1 , wherein the first media agent is implemented at a secondary storage computing system included in a secondary storage environment and the first client computing system is included in a primary storage environment of the information management system.

9. The computer-implemented method of claim 8 , wherein at least said generating the first transaction log file is performed by a core media agent at the first client computing system that is separate from the first media agent.

10. The computer-implemented method of claim 1 , wherein a second transaction log file associated with backup of a second client computing system is provided to the second media agent, and wherein the second transaction log file is replicated to the first media agent.

11. The computer-implemented method of claim 1 , further comprising:
detecting failure of a secondary storage computing system hosting the first media agent;
configuring a computing system as a replacement secondary storage computing system;
replicating the first transaction log from the second media agent to a replacement media agent hosted by the replacement secondary storage computing system; and
registering the replacement media agent with a storage manager of the information management system.

12. The computer-implemented method of claim 1 , further comprising:
receiving a request to access a backup index associated with the job identifier;
determining that a set of transaction log files associated with the job identifier exist, the set of transaction log files including the first transaction log file;
updating the backup index based at least in part on the set of transaction log files to obtain an updated backup index; and
providing access to the updated backup index responsive to the request to access the backup index.

13. The computer-implemented method of claim 12 , further comprising flushing the set of transaction log files after updating the backup index.

14. A system for reducing storage manager overhead associated with secondary storage management of an information management system, the system comprising:
a log manager implemented by one or more hardware processors of a client computing system, the log manager configured to:
access first backup metadata associated with a first backup of data from a primary storage device of a client computing system to a network storage system as part of a backup process at a first time, the first backup performed using a cached job identifier obtained from a storage manager at a second time that is earlier than the first time, wherein the network storage system is external to the information management system;
generate a first transaction log file based on the first backup metadata; and
provide the first transaction log file to a first media agent at a first secondary storage computing system of the information management system; and

the first media agent implemented by one or more hardware processors of the secondary storage computing system, the first media agent configured to maintain the first transaction log file at a plurality of media agents by replicating the first transaction log to at least a second media agent of the information management system hosted by a second secondary storage computing system, wherein the second media agent is selected from the plurality of media agents based at least in part on an identity of the client computing system or a job identifier associated with the first transaction log file.

15. The system of claim 14 , wherein the first media agent is further configured to:
determine an occurrence of a trigger to update a backup index;
access a set of transaction log files including the first transaction log file;
update the backup index based on the set of transaction log files;
replicate the backup index within a distributed index storage system; and
flush the set of transaction log files.

16. The system of claim 15 , wherein the first media agent is further configured to cause the set of transaction log files to be flushed from the second media agent.

17. The system of claim 15 , wherein the first media agent determines the set of transaction log files to access based at least in part on the job identifier associated with the client computing system.

18. The system of claim 14 , further comprising a storage manager configured to:
detect failure of the secondary storage computing system implementing the first media agent;
configure a computing system as a

Assignees

• Commvault Systems Inc

Inventors

• Bedadala Pavan Kumar Reddy
• PRADHAN Manoj Kumar
• Mishra Hemant
• Venugopalsamy Gopikannan

Classifications

• G06F11/1456

  Hardware arrangements for backup · CPC title

• G06F2201/835

  Timestamp · CPC title

• G06F11/1474

  in transactions (updating of structured data in databases G06F16/23) · CPC title

• G06F2201/815

  Virtual · CPC title

• G06F16/27

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title