Source-side accelerated indexing for data protection search

Patent metadata

Field	Value
Publication number	US-11429491-B2
Application number	US-201816172563-A
Country	US
Kind code	B2
Filing date	Oct 26, 2018
Priority date	Oct 26, 2018
Publication date	Aug 30, 2022
Grant date	Aug 30, 2022

Abstract

A method, apparatus, and system for storing indexed metadata associated with backed up data is provided. The operations comprising: for each first data item stored at a first client device to be backed up, extracting, at the first client device, an associated first metadata item; transforming, at the first client device, the first metadata item based on a predefined metadata definition; associating, at the first client device, a first client identifier and a backup identifier with the transformed first metadata item; and transferring the transformed first metadata item from the first client device to a metadata bus of a search cluster.

First claim

What is claimed is:

1. A computer-implemented method for storing indexed metadata associated with backed up data, comprising:
for each first data item stored at a first client device to be backed up, extracting, at the first client device, an associated first metadata item;
transforming, at the first client device, the first metadata item based on a predefined metadata definition;
associating, at the first client device, a first client identifier and a backup identifier with the transformed first metadata item;
transferring the transformed first metadata item from the first client device to a metadata bus of a search cluster; and
increasing a number of search nodes to accelerate handling of metadata items stored at the metadata bus.

2. The method of claim 1 , wherein the first data item to be backed up is a file, and wherein transforming the first metadata item based on a predefined metadata definition comprises generating a metadata item for the file according to a predefined format using the extracted associated first metadata item associated with the file.

3. The method of claim 1 , further comprising transferring the first data item to be backed up from the first client device to a backup appliance.

4. The method of claim 1 , wherein a search node of the search cluster retrieves the transformed first metadata item from the metadata bus and transfers the transformed first metadata item to an index engine/platform for storage, and wherein the transformed first metadata item stored at the index engine/platform is user searchable, the transformed first metadata item retrieved by the search node being removed from the metadata bus, a number of search nodes being reduced if a number of the transformed metadata items in a metadata queue is below a predefined threshold.

5. The method of claim 1 , further comprising combining, at the first client device, a plurality of transformed first metadata items associated with the first client identifier and a same backup identifier into a single combined transformed first metadata item; and determining that the metadata bus is not full.

6. The method of claim 1 , further comprising decreasing a number of search nodes in response to a determination that the metadata bus is not full.

7. The method of claim 1 , further comprising:
for each second data item stored at a second client device to be backed up, extracting, at the second client device, an associated second metadata item;
transforming, at the second client device, the second metadata item based on the predefined metadata definition;
associating, at the second client device, a second client identifier and the backup identifier with the transformed second metadata item; and
transferring the transformed second metadata item from the second client device to the metadata bus of the search cluster.

8. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for storing indexed metadata associated with backed up data, the operations comprising:
for each first data item stored at a first client device to be backed up, extracting, at the first client device, an associated first metadata item;
transforming, at the first client device, the first metadata item based on a predefined metadata definition;
associating, at the first client device, a first client identifier and a backup identifier with the transformed first metadata item;
transferring the transformed first metadata item from the first client device to a metadata bus of a search cluster; and
increasing a number of search nodes to accelerate handling of metadata items stored at the metadata bus.

9. The non-transitory machine-readable medium of claim 8 , wherein the first data item to be backed up is a file, and wherein transforming the first metadata item based on a predefined metadata definition comprises generating a metadata item for the file according to a predefined format using the extracted associated first metadata item associated with the file.

10. The non-transitory machine-readable medium of claim 8 , the operations further comprising transferring the first data item to be backed up from the first client device to a backup appliance.

11. The non-transitory machine-readable medium of claim 8 , wherein a search node of the search cluster retrieves the transformed first metadata item from the metadata bus and transfers the transformed first metadata item to an index engine/platform for storage, and wherein the transformed first metadata item stored at the index engine/platform is user searchable, the transformed first metadata item retrieved by the search node being removed from the metadata bus, a number of search nodes being reduced if a number of the transformed metadata items in a metadata queue is below a predefined threshold.

12. The non-transitory machine-readable medium of claim 8 , the operations further comprising combining, at the first client device, a plurality of transformed first metadata items associated with the first client identifier and a same backup identifier into a single combined transformed first metadata item; and determining that the metadata bus is not full.

13. The non-transitory machine-readable medium of claim 8 , the operations further comprising decreasing a number of search nodes in response to a determination that the metadata bus is not full.

14. The non-transitory machine-readable medium of claim 8 , the operations further comprising:
for each second data item stored at a second client device to be backed up, extracting, at the second client device, an associated second metadata item;
transforming, at the second client device, the second metadata item based on the predefined metadata definition;
associating, at the second client device, a second client identifier and the backup identifier with the transformed second metadata item; and
transferring the transformed second metadata item from the second client device to the metadata bus of the search cluster.

15. A data processing system, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations for storing indexed metadata associated with backed up data, the operations including:
for each first data item stored at a first client device to be backed up, extracting, at the first client device, an associated first metadata item;
transforming, at the first client device, the first metadata item based on a predefined metadata definition;
associating, at the first client device, a first client identifier and a backup identifier with the transformed first metadata item;
transferring the transformed first metadata item from the first client device to a metadata bus of a search cluster; and
increasing a number of search nodes to accelerate handling of metadata items stored at the metadata bus.

16. The data processing system of claim 15 , wherein the first data item to be backed up is a file, and wherein transforming the first metadata item based on a predefined metadata definition comprises generating a metadata item for the file according to a predefined format using the extracted associated first metadata item associated with the file.

17. The data processing system of claim 15 , the operations further comprising transferring the first data item to be backed up from the first client device to a backup appliance.

18. The data processing system of claim 15 , wherein a search node of the search cluster retrie

Assignees

• Emc Ip Holding Co Llc

Inventors

• Yu Jing
• Liao mengze
• Guo Yongsheng
• Brenner Adam
• Zhang Ming

Classifications

• G06F16/13

  File access structures, e.g. distributed indices (arrangements of input from, or output to, record carriers G06F3/06) · CPC title

• G06F11/1435

  using file system or storage system metadata · CPC title

• G06F11/1464

  for networked environments · CPC title

• G06F16/14 Primary

  Details of searching files based on file metadata · CPC title

• G06F11/1451 Primary

  by selection of backup contents · CPC title