Method and system for generating backup of a large file and optimizing resource utilization

Patent metadata

Field	Value
Publication number	US-12007850-B2
Application number	US-202217687333-A
Country	US
Kind code	B2
Filing date	Mar 4, 2022
Priority date	Mar 4, 2022
Publication date	Jun 11, 2024
Grant date	Jun 11, 2024

Abstract

A method for performing data backup includes: receiving a backup request specifying a file where the file is stored in a shared storage. Additionally, in response to the backup request: retrieving, by a primary data node, properties of the file; determining, by the primary data node, resource availability for plurality of secondary data nodes where the primary data node and the plurality of secondary data nodes is operatively connected to the shared storage and a backup storage; logically dividing, by the primary data node, the file into a plurality of segments using at least the properties of the file; and coordinating, by the primary data node, the backup of the file using the plurality of segments and the resource availability of the plurality of secondary data nodes.

First claim

What is claimed is:

1. A method for performing data backup, the method comprising:
receiving a backup request specifying a file, wherein the file is stored in a shared storage;
in response to the backup request:
retrieving, by a primary data node, properties of the file;
determining, by the primary data node, resource availability for plurality of secondary data nodes, wherein the primary data node and the plurality of secondary data nodes are operatively connected to the shared storage and a backup storage;
logically dividing, by the primary data node, the file into a plurality of segments using at least the properties of the file; and
coordinating, by the primary data node, the backup of the file using the plurality of segments and the resource availability of the plurality of secondary data nodes,
wherein coordinating the backup of the file using the plurality of segments and the resource availability of the plurality of secondary data nodes comprises:
allocating a first set of segments from the plurality of segments to a first secondary data node of the plurality of secondary data nodes;
allocating a second set of segments from the plurality of segments to a second secondary data node of the plurality of secondary data nodes;
receiving, by the primary data node, a first completion notification from the first secondary data node and a second completion notification from the second secondary data node;
generate, in response to receiving the first completion notification and the second completion notification, aggregated metadata specifying storage locations of all the plurality of segments in the backup storage; and
sending, by the primary data node, the aggregated metadata to the backup storage,

wherein the first secondary data node backs up the first set of segments,
wherein the second secondary data node backs up the second set of segments, and
wherein the file comprises the first set of segments and the second set of segments.

2. The method of claim 1 , further comprising, in response to the backup request:
obtaining a snapshot comprising the file,
wherein the snapshot is stored on the shared storage,
wherein the snapshot is accessible to the primary data node and the plurality of secondary nodes, and
wherein the snapshot is used by plurality of secondary data nodes to back up the file.

3. The method of claim 1 , wherein logically dividing the file into the plurality of segments comprises:
obtaining a predetermined segment size; and
logically dividing the file based on the predetermined segment size.

4. The method of claim 1 , wherein the first completion notification comprises first metadata specifying first storage locations of the first set of segments in the backup storage and wherein the second completion notification comprises second metadata specifying second storage locations of the second set of segments in the backup storage.

5. A production host for hosting virtual machines, comprising:
a persistent storage storing:
a data node monitoring information repository;
a data limit information repository;
a bandwidth limit information repository; and
a primary data node monitoring information;

an application agent programmed to:
receive a backup request specifying a file, wherein the file is stored in a shared storage; and
in response to the backup request:
retrieve, by a primary data node, properties of the file;
determine, by the primary data node, resource availability for plurality of secondary data nodes, wherein the primary data node and the plurality of secondary data nodes is operatively connected to the shared storage and a backup storage;
logically divide, by the primary data node, the file into a plurality of segments using at least the properties of the file; and
coordinate, by the primary data node, the backup of the file using the plurality of segments and the resource availability of the plurality of secondary data nodes,
wherein coordinating the backup of the file using the plurality of segments and the resource availability of the plurality of secondary data nodes comprises:
allocating a first set of segments from the plurality of segments to a first secondary data node of the plurality of secondary data nodes;
allocating a second set of segments from the plurality of segments to a second secondary data node of the plurality of secondary data nodes;
receiving, by the primary data node, a first completion notification from the first secondary data node and a second completion notification from the second secondary data node;
generate, in response to receiving the first completion notification and the second completion notification, aggregated metadata specifying storage locations of all the plurality of segments in the backup storage; and
sending, by the primary data node, the aggregated metadata to the backup storage,

wherein the first secondary data node backs up the first set of segments,
wherein the second secondary data node backs up the second set of segments, and
wherein the file comprises the first set of segments and the second set of segments.

6. The production host of claim 5 , wherein the application agent is further programmed to, in response to the backup request:
obtain a snapshot comprising the file,
wherein the snapshot is stored on the shared storage,
wherein the snapshot is accessible to the primary data node and the plurality of secondary nodes, and
wherein the snapshot is used by plurality of secondary data nodes to back up the file.

7. The production host of claim 5 , wherein logically dividing the file into the plurality of segments comprises:
obtaining a predetermined segment size; and
logically dividing the file based on the predetermined segment size.

8. A non-transitory computer readable medium (CRM) comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method for hosting virtual machines, the method comprising:
receiving a backup request specifying a file, wherein the file is stored in a shared storage; and
in response to the backup request:
retrieving, by a primary data node, the properties of the file from a snapshot;
determining, by the primary data node, querying resource availability for a plurality of a secondary data nodes, by the primary data node, wherein the primary data node and the plurality of secondary data nodes is operatively connected to the shared storage and a backup storage;
logically dividing, by the primary data node, the file into a plurality of segments, by the primary data node using at least the properties of the file;
coordinating, by the primary data node, the backup of the file using the plurality of segments and the resource availability of the plurality of secondary data nodes, wherein coordinating the backup of the file using the plurality of segments and the resource availability of the plurality of secondary data nodes comprises:
allocating a first set of segments from the plurality of segments to a first secondary data node of the plurality of secondary data nodes;
allocating a second set of segments from the plurality of segments to a second secondary data node of the plurality of secondary data nodes;
receiving, by the primary data node, a first completion notification from the first secondary data node and a second completion notification from the second secondary data node;
generate, in response to receiving the first completion notification and the second completion notification, aggregated metadata specifying storage locations of all the plurality of segments in the backup storage; and
sending, by the primary data node, the aggregated metadata to the backup storage,

wherein the first secondary d

Assignees

• Dell Products Lp

Inventors

• Yadav Sunil
• Chopra Shelesh
• Bansal Aaditya Rakesh
• Sharma Manish

Classifications

• G06F11/1451

  by selection of backup contents · CPC title

• G06F11/1435

  using file system or storage system metadata · CPC title

• G06F2201/84

  Using snapshots, i.e. a logical point-in-time copy of the data · CPC title

• G06F11/1469

  Backup restoration techniques · CPC title

• G06F11/3006

  where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50) · CPC title