Synchronization of metadata in a distributed storage system

Patent metadata

Field	Value
Publication number	US-11934356-B2
Application number	US-202217708312-A
Country	US
Kind code	B2
Filing date	Mar 30, 2022
Priority date	Dec 7, 2017
Publication date	Mar 19, 2024
Grant date	Mar 19, 2024

Abstract

A client machine writes to and reads from a virtual disk on a remote storage platform. Metadata is generated and stored in replicas on different metadata nodes of the storage platform. A modified log-structured merge tree is used to store and compact string-sorted tables of metadata. During file storage and compaction, a consistent file identification scheme is used across all metadata nodes. A fingerprint file is calculated for each SST (metadata) file on disk that includes hash values corresponding to regions of the SST file. To synchronize, the fingerprint files of two SST files are compared, and if any hash values are missing from a fingerprint file, then the key-value-timestamp triples corresponding to these missing hash values are sent to the SST file that is missing them. The SST file is compacted with the missing triples to create a new version of the SST file. The synchronization is bi-directional.

First claim

We claim:

1. A system comprising:
a plurality of computer nodes, wherein each computer node in the plurality of computer nodes comprises at least one hardware processor and one or more data storage drives, wherein a first computer node among the plurality of computer nodes is configured to:
retrieve, from one of the plurality of computer nodes, a first fingerprint file that includes a plurality of first hash values,
wherein each first hash value in the plurality of first hash values corresponds to a first region of a first metadata file stored at the first computer node among the plurality of computer nodes,
wherein the first metadata file includes a first plurality of key-value-timestamp triples each of which comprises a portion of metadata that pertains to particular data that was stored into a computer node among the plurality of computer nodes of the system, and
wherein each first region of the first metadata file comprises at least part of a key-value-timestamp triple among the first plurality of key-value-timestamp triples;

identify a second metadata file as a replica of the first metadata file,
wherein the second metadata file is stored at a second computer node among the plurality of computer nodes,
wherein the second metadata file includes a second plurality of key-value-timestamp triples each of which comprises a portion of metadata that pertains to the particular data that was stored into a computer node among the plurality of computer nodes;

retrieve, from one of the plurality of computer nodes, a second fingerprint file that includes a plurality of second hash values,
wherein each second hash value corresponds to a second region of the second metadata file, and
wherein each second region of the second metadata file comprises at least part of a key-value-timestamp triple among a second plurality of key-value-timestamp triples in the second metadata file;

compare the plurality of first hash values with the plurality of second hash values;
determine that the plurality of second hash values lacks at least one first hash value that is present in the plurality of first hash values, wherein the first metadata file comprises at least one key-value-timestamp triple that corresponds to the at least one first hash value lacking from the plurality of second hash values;
cause the second metadata file to be updated with the at least one key-value-timestamp triple that corresponds to the at least one first hash value lacking from the plurality of second hash values; and
wherein one or more of the first computer node and the second computer node is configured to: determine whether all second hash values in the plurality of second hash values are present among the plurality of first hash values.

2. The system of claim 1 , wherein one or more of the first computer node and the second computer node is configured to:
determine that the plurality of first hash values lacks at least one second hash value that is present in the plurality of second hash values, wherein the second metadata file comprises at least one key-value-timestamp triple that corresponds to the at least one second hash value lacking from the plurality of first hash values; and
cause the first metadata file to be updated with the at least one key-value-timestamp triple that corresponds to the at least one second hash value lacking from the plurality of first hash values.

3. The system of claim 2 , wherein updating of the first metadata file and updating of the second metadata file synchronizes bi-directionally, between the first computer node and the second computer node, metadata that pertains to the particular data.

4. The system of claim 1 , wherein the at least one first hash value lacking from the plurality of second hash values corresponds to at least one missing region of the second metadata file.

5. The system of claim 1 , wherein the second computer node is configured to create a new version of the second metadata file by compacting the second metadata file as updated with the at least one key-value-timestamp triple that corresponds to the at least one first hash value lacking from the plurality of second hash values.

6. The system of claim 1 , wherein the first metadata file and the second metadata file are located on different computer nodes among the plurality of computer nodes, and wherein the first metadata file and the second metadata file have a same file identifier, and wherein the same file identifier indicates that the second metadata file is a replica of the first metadata file.

7. The system of claim 1 , wherein the particular data was stored into a given computer node among the plurality of computer nodes, wherein the given computer node is associated with the first metadata file;
wherein the particular data was also stored into an other computer node among the plurality of computer nodes, wherein the other computer node is associated with the second metadata file; and
wherein the first computer node is distinct from the given computer node and does not comprise the particular data.

8. The system of claim 1 , wherein the first fingerprint file and the second fingerprint file are retrieved from a same computer node among the plurality of computer nodes.

9. The system of claim 1 , wherein each first hash value is part of a start-length-hash value triple that uniquely corresponds to a first region of the first metadata file as stored on a data storage drive at the first computer node.

10. The system of claim 1 , wherein the first metadata file is organized as a string-sorted-table (SST) comprising the first plurality of key-value-timestamp triples sorted by key; and wherein the second metadata file is organized as a string-sorted-table (SST) comprising the second plurality of key-value-timestamp triples sorted by key.

11. A system comprising:
a plurality of computer nodes, wherein each computer node in the plurality of computer nodes comprises at least one hardware processor and one or more data storage drives, wherein a first computer node among the plurality of computer nodes is configured to:
retrieve a plurality of first hash values from a first fingerprint file,
wherein each first hash value is part of a start-length-hash value triple that uniquely corresponds to a first region of a first metadata file, which is stored at the first computer node among the plurality of computer nodes,
wherein the first metadata file includes a first plurality of key-value-timestamp triples each of which comprises a portion of metadata that pertains to particular data that was stored into a computer node among the plurality of computer nodes of the system, and
wherein each first region of the first metadata file comprises at least part of a key-value-timestamp triple among the first plurality of key-value-timestamp triples;

determine that a second metadata file is a replica of the first metadata file,
wherein the second metadata file is stored at a second computer node among the plurality of computer nodes,
wherein the second metadata file includes a second plurality of key-value-timestamp triples each of which comprises a portion of metadata that pertains to the particular data that was stored into a computer node among the plurality of computer nodes;

retrieve a plurality of second hash values from a second fingerprint file,
wherein each second hash value corresponds to a second region of the second metadata file,
wherein each second hash value is part of a start-length-hash value triple that uniquely corresponds to a second region of the second metadata file as stored on a data storage drive at a second computer node among the plurality of computer nodes, and
wherein each second region of the s

Assignees

• Commvault Systems Inc

Inventors

• Lakshman Avinash
• Camargos Lasaro
• Jain Deepak

Classifications

• G06F16/178 Primary

  Techniques for file synchronisation in file systems · CPC title

• G06F11/1464

  for networked environments · CPC title

• G06F16/137

  Hash-based (content-based indexing of textual data G06F16/31) · CPC title

• G06F16/24573

  using data annotations, e.g. user-defined metadata · CPC title

• G06F16/2471

  Distributed queries · CPC title