Monitoring of nodes within a distributed storage environment

Patent metadata

Field	Value
Publication number	US-11301334-B2
Application number	US-201916550064-A
Country	US
Kind code	B2
Filing date	Aug 23, 2019
Priority date	Mar 9, 2016
Publication date	Apr 12, 2022
Grant date	Apr 12, 2022

Abstract

In certain systems disclosed herein, a distributed data monitoring and management system is provided that can replicate a distributed storage environment. The distributed data monitoring and management system can intelligently and automatically configure data access nodes to form a structure that matches the distributed storage environment. By matching the structure of the distributed storage environment, the distributed structure of the data may be maintained, enabling the data to be backed up from and/or restored to the distributed storage environment and/or migrated to another distributed storage environment without altering the distribution of the data. Further, embodiments herein enable the monitoring of nodes within the system and transfer of data from a non-distributed environment to a distributed storage environment. Thus, in some cases, an entity can migrate data from a local storage structure to a network-based distributed storage structure.

First claim

What is claimed is:

1. A distributed system for monitoring nodes in a distributed computing environment, the distributed system comprising:
a control agent implemented on a control computing system comprising memory and one or more processors; and
a plurality of data access nodes implemented, respectively, on a plurality of computing systems each comprising memory and one or more processors,
wherein the control agent is programmed to:
access configuration information for a distributed storage environment, wherein the distributed storage environment comprises a plurality of storage nodes;
determine a distributed file system type for the distributed storage environment based at least in part on the configuration information;
determine a total number of storage nodes within the distributed storage environment based at least in part on the configuration information;
make a determination on how to assign the plurality of data access nodes to transfer data from the plurality of storage nodes by distributing the plurality of data access nodes among the plurality of storage nodes based at least in part on the distributed file system type and the total number of storage nodes; and
assign the plurality of data access nodes to the plurality of storage nodes based on the determination.

2. The distributed system of claim 1 , wherein the plurality of data access nodes comprise a plurality of data agents and the plurality of computing systems are part of a primary storage subsystem.

3. The distributed system of claim 1 , wherein the plurality of data access nodes comprise a plurality of media agents and the plurality of computing systems are part of a secondary storage subsystem.

4. The distributed system of claim 1 , wherein, in response to determining that a data access node from the plurality of data access nodes is inaccessible, the control agent is further programmed to re-distribute the plurality of data access nodes among the plurality of storage nodes.

5. The distributed system of claim 1 , wherein, in response to determining that a data access node from the plurality of data access nodes is inaccessible, the control agent is further programmed to:
allocate a new data access node to the plurality of data access nodes; and
re-distribute the plurality of data access nodes with the new data access node among the plurality of storage nodes.

6. The distributed system of claim 1 , wherein, in response to determining that a storage node from the plurality of storage nodes is inaccessible, the control agent is further programmed to reassign data access nodes from the plurality of data access nodes assigned to the storage node to one or more other storage nodes from the plurality of storage nodes that store at least a subset of data blocks stored on the storage node.

7. The distributed system of claim 1 , wherein, in response to determining that a storage node from the plurality of storage nodes is inaccessible, the control agent is further programmed to reassign data access nodes from the plurality of data access nodes assigned to the storage node to a mirror node that mirrors data stored on the storage node.

8. The distributed system of claim 1 , wherein the control agent is further programmed to distribute the plurality of data access nodes among the plurality of storage nodes based at least in part on a size of files stored at the plurality of storage nodes.

9. The distributed system of claim 1 , wherein the plurality of data access nodes comprises a plurality of heterogeneous data access nodes with differing backup capabilities.

10. The distributed system of claim 9 , wherein the control agent is further programmed to distribute the plurality of data access nodes among the plurality of storage nodes by distributing the plurality of heterogeneous data access nodes based at least in part on the differing backup capabilities of the plurality of heterogeneous data access nodes.

11. A computer-implemented method of for monitoring nodes in a distributed computing environment, the computer-implemented method comprising:
as implemented by a control agent comprising one or more hardware processors and configured with specific computer-executable instructions, accessing configuration information for a distributed storage environment from a configuration node of the distributed storage environment, wherein the distributed storage environment comprises a plurality of storage nodes;
determining a distributed file system type for the distributed storage environment based at least in part on the configuration information;
determining a total number of storage nodes within the distributed storage environment based at least in part on the configuration information;
determining how to assign a plurality of data access nodes to back up the plurality of storage nodes by distributing the plurality of data access nodes among the plurality of storage nodes based at least in part on the distributed file system type and the total number of storage nodes, wherein the plurality of data access nodes are implemented, respectively, on a plurality of computing systems; and
assigning the plurality of data access nodes to the plurality of storage nodes based on the determining how to assign the plurality of data access nodes.

12. The computer-implemented method of claim 11 , wherein the plurality of data access nodes comprises a plurality of data agents and the plurality of computing systems are part of a primary storage subsystem.

13. The computer-implemented method of claim 11 , wherein the plurality of data access nodes comprises a plurality of media agents and the plurality of computing systems are part of a secondary storage subsystem.

14. The computer-implemented method of claim 11 , wherein, in response to determining that a data access node from the plurality of data access nodes is inaccessible, the computer-implemented method further comprises re-distributing the plurality of data access nodes among the plurality of storage nodes.

15. The computer-implemented method of claim 11 , wherein, in response to determining that a data access node from the plurality of data access nodes is inaccessible, the computer-implemented method further comprises:
allocating a new data access node to the plurality of data access nodes; and
re-distributing the plurality of data access nodes with the new data access node among the plurality of storage nodes.

16. The computer-implemented method of claim 11 , wherein, in response to determining that a storage node from the plurality of storage nodes is inaccessible, the computer-implemented method further comprises reassigning data access nodes from the plurality of data access nodes assigned to the storage node to one or more other storage nodes from the plurality of storage nodes that store at least a subset of data blocks stored on the storage node.

17. The computer-implemented method of claim 11 , wherein, in response to determining that a storage node from the plurality of storage nodes is inaccessible, the computer-implemented method further comprises reassigning data access nodes from the plurality of data access nodes assigned to the storage node to a mirror node that mirrors data stored on the storage node.

18. The computer-implemented method of claim 11 , further comprising distributing the plurality of data access nodes among the plurality of storage nodes based at least in part on a size of files stored at the plurality of storage nodes.

19. The computer-implemented method of claim 11 , wherein the plurality of data access

Assignees

• Commvault Systems Inc

Inventors

• PRADHAN Manoj Kumar
• Kumarasamy Paramasivam
• Zakharkin Dmitriy Borisovich
• Duraisamy Arun Prabu

Classifications

• G06F11/1464 Primary

  for networked environments · CPC title

• H04L67/1097 Primary

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• H04L67/60

  Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources (admission control or resource allocation H04L47/70) · CPC title

• H04L67/01

  Protocols · CPC title

• H04L69/40

  for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection (management of faults, events, alarms or notifications in data switching networks H04L41/06) · CPC title