Historical network data-based scanning thread generation

Patent metadata

Field	Value
Publication number	US-2018121297-A1
Application number	US-201615341878-A
Country	US
Kind code	A1
Filing date	Nov 2, 2016
Priority date	Nov 2, 2016
Publication date	May 3, 2018
Grant date	—

Abstract

Embodiments disclosed herein address the need to more efficiently backup a network-based storage environment that may be remote from a primary storage environment. For example, embodiments herein can provide a more efficient backup of a storage managed by a third-party entity. To improve the backup process, embodiments herein may optimize the number of scanning threads that are used to identify files that are to be backed up by, for example, analyzing the characteristics of the network and/or the network storage system to determine a number of scanning threads that will enable faster scanning of the network storage system while at the same time not overburden or be slowed down by a network between the network-based storage environment and the primary storage environment.

First claim

What is claimed is:

1 . A computer-implemented method of performing multi-threaded scanning of a network storage system, the computer-implemented method comprising:
as implemented by a data agent within a remote storage access system comprising one or more hardware processors and configured with specific computer-executable instructions,
determining whether historical network characteristics of a network used to communicate with a network storage system exist; and
in response to determining that the historical network characteristics exist:
accessing the historical network characteristics;
selecting an amount of scanning threads to initiate based at least in part on the historical network characteristics, wherein the scanning threads are configured to scan a network storage repository of the network storage system to identify files to backup;
triggering initiation of the amount of the scanning threads at the network storage system; and
initiating scanning of the network storage repository using the scanning threads to identify the files to backup.

2 . The computer-implemented method of claim 1 , wherein scanning the network storage repository comprises traversing a file structure at the network storage repository.

3 . The computer-implemented method of claim 1 , further comprising:
receiving a scan result from the network storage system corresponding to a result of scanning the network storage repository; and initiating backup of the network storage repository based at least in part on the scan result.

4 . The computer-implemented method of claim 3 , wherein the scan result identifies files to be backed up and a location of the files at the network storage repository.

5 . The computer-implemented method of claim 3 , further comprising:
receiving files to be backed up responsive to the initiating backup of the network storage repository; and providing the files to a media agent for backup at a secondary storage system.

6 . The computer-implemented method of claim 1 , wherein the historical network characteristics are determined based at least in part on communication with the network storage system during performance of a particular number of backup processes occurring during a particular time period prior to the accessing of the historical network characteristics.

7 . The computer-implemented method of claim 1 , wherein the historical network characteristics are determined based at least in part on the transmission of a number of test packets during a time period occurring prior to the accessing of the historical network characteristics.

8 . The computer-implemented method of claim 1 , further comprising determining current network characteristics of the network, wherein the selecting the amount of the scanning threads is based at least in part on an aggregation of the historical network characteristics and the current network characteristics.

9 . The computer-implemented method of claim 8 , wherein the historical network characteristics and the current network characteristics are weighted differently in the aggregation of the historical network characteristics and the current network characteristics.

10 . The computer-implemented method of claim 8 , further comprising updating the historical network characteristics based on the current network characteristics for subsequent scanning of the network storage repository.

11 . The computer-implemented method of claim 1 , wherein, in response to determining that the historical network characteristics do not exist, the computer-implemented method further comprises:
determining current network characteristics of the network; and selecting the amount of the scanning threads based at least in part on the current network characteristics.

12 . A system for performing multi-threaded scanning of a network storage system, the system comprising:
a remote storage access system implemented in computer hardware, the remote storage access system configured to:
determine whether historical network characteristics of a network used to communicate with a network storage system exist; and
in response to determining that the historical network characteristics exist:
access the historical network characteristics;
select an amount of scanning threads to initiate based at least in part on the historical network characteristics, wherein the scanning threads are configured to scan a network storage repository of the network storage system to identify files to backup;
trigger initiation of the amount of the scanning threads at the network storage system; and
initiate scanning of the network storage repository using the scanning threads to identify the files to backup.

13 . The system of claim 12 , wherein the remote storage access system is further configured to:
receive a scan result from the network storage system corresponding to a result of scanning the network storage repository; and initiate backup of the network storage repository based at least in part on the scan result.

14 . The system of claim 13 , wherein the remote storage access system is further configured to:
receive files to be backed up responsive to the initiating backup of the network storage repository; and provide the files to a media agent for backup at a secondary storage system.

15 . The system of claim 12 , wherein the historical network characteristics are determined based at least in part on communication with the network storage system during performance of a particular number of backup processes occurring during a particular time period prior to the accessing of the historical network characteristics.

16 . The system of claim 12 , wherein the historical network characteristics are determined based at least in part on the transmission of a number of test packets during a time period occurring prior to the accessing of the historical network characteristics.

17 . The system of claim 12 , wherein the remote storage access system is further configured to determine current network characteristics of the network, wherein the selecting the amount of the scanning threads is based at least in part on an aggregation of the historical network characteristics and the current network characteristics.

18 . The system of claim 17 , wherein the historical network characteristics and the current network characteristics are weighted differently in the aggregation of the historical network characteristics and the current network characteristics.

19 . The system of claim 17 , wherein the remote storage access system is further configured to update the historical network characteristics based on the current network characteristics for subsequent scanning of the network storage repository.

20 . The system of claim 12 , wherein, in response to determining that the historical network characteristics do not exist, the remote storage access system is further configured to:
determine current network characteristics of the network; and select the amount of the scanning threads based at least in part on the current network characteristics.

Assignees

• Commvault Systems Inc

Inventors

• Swallow Douglas Cary
• PRADHAN Manoj Kumar
• Zakharkin Dmitriy Borisovich

Classifications

• G06F11/1464 Primary

  for networked environments · CPC title

• G06F2201/815

  Virtual · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F11/2097

  maintaining the standby controller/processing unit updated (initialisation or re-synchronisation thereof G06F11/1658 and subgroups) · CPC title

• G06F11/3072

  where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting · CPC title