Fast migration of metadata

Patent metadata

Field	Value
Publication number	US-2021181968-A1
Application number	US-202017111076-A
Country	US
Kind code	A1
Filing date	Dec 3, 2020
Priority date	Aug 10, 2018
Publication date	Jun 17, 2021
Grant date	—

Abstract

One or more buckets of key-value pairs of a first node of a distributed storage system are selected to be migrated to a second node of the distributed storage system. One or more underlying database files corresponding to the one or more selected buckets are identified. The one or more identified underlying database files are directly copied from a storage of a first node to a storage of the second node. The copied underlying database files are linked in a database of the second node to implement the one or more selected buckets in the second node.

First claim

What is claimed is:

1 . (canceled)

2 . A system, comprising:
a first storage node configured to:
determine one or more underlying database files that correspond to one or more selected buckets of key-value pairs;
directly copy to a second storage node the one or more underlying database files that correspond to one or more selected buckets of key-value pairs, wherein updates to the one or more underlying database files are suspended while the one or more underlying database files are being sent to the second storage node; and
the second storage node configured to:
receive the one or more underlying database files; and
update a local key-value store based on the one or more received underlying database files.

3 . The system of claim 2 , wherein the first storage node is configured to:
log the updates to the one or more underlying database files after the updates to the one or more database files are suspended while the one or more underlying database files are being sent to the second storage node; and send the logged updates to the second storage node.

4 . The system of claim 3 , wherein the second storage node is configured to apply the logged updates to the one or more received underlying database files.

5 . The system of claim 3 , wherein the first storage node is configured to inspect a log after the one or more underlying database files are copied to the second storage node.

6 . The system of claim 5 , wherein the first storage node is configured to determine whether a number of entries included in the log is less than a threshold number of updates.

7 . The system of claim 6 , wherein in the event the number of entries included in the log is less than the threshold number of updates, the first storage node is configured to:
suspend updates to the one or more selected buckets of key-value pairs; and send the log to the second storage node.

8 . The system of claim 6 , wherein in the event the number of entries included in the log is not less than the threshold number of updates, the first storage node is configured to determine whether a retry threshold has been exceeded.

9 . The system of claim 8 , wherein in the event the retry threshold has not been exceeded, the first storage node is configured to:
generate a new log; and send the log to the second storage node.

10 . The system of claim 8 , wherein in the event the retry threshold has been exceeded, the first storage node is configured to:
suspend updates to the one or more selected buckets of key-value pairs; and send the log to the second storage node.

11 . The system of claim 2 , wherein the first storage node is configured to suspend the updates to the one or more database files.

12 . The system of claim 2 , wherein the one or more selected buckets of key-value pairs correspond to one or more buckets of key-value pairs that are accessed at a frequency greater than a threshold.

13 . The system of claim 2 , wherein the second storage node is a new storage node associated within the system.

14 . The system of claim 2 , wherein the local key-value store is a log structured merge database.

15 . The system of claim 2 , wherein the first storage node is configured to update a master table that indicates which storage node of a plurality of storage nodes is to handle a file operation for a particular key-value pair.

16 . The system of claim 15 , wherein entries included in the master table that correspond to the key-value pairs included in the one or more selected buckets are updated to reference the second storage node in place of the first storage node.

17 . A method, comprising:
determining, by a first storage node, one or more underlying database files that correspond to one or more selected buckets of key-value pairs; and directly copying, from the first storage node to a second storage node, the one or more underlying database files that correspond to one or more selected buckets of key-value pairs, wherein updates to the one or more underlying database files are suspended while the one or more underlying database files are being sent to the second storage node, wherein the second storage node receives the one or more underlying database files and updates a local key-value store based on the one or more received underlying database files.

18 . The method of claim 17 , wherein the one or more selected buckets of key-value pairs correspond to one or more buckets of key-value pairs that are accessed at a frequency greater than a threshold.

19 . The method of claim 17 , further comprising:
logging the updates to the one or more underlying database files after the updates to the one or more database files are suspended while the one or more underlying database files are being sent to the second storage node; and sending the logged updates to the second storage node.

20 . The method of claim 19 , wherein the second storage node applies the logged updates to the one or more received underlying database files.

21 . A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:
determining, by a first storage node, one or more underlying database files that correspond to one or more selected buckets of key-value pairs; and directly copying, from the first storage node to a second storage node, the one or more underlying database files that correspond to one or more selected buckets of key-value pairs, wherein updates to the one or more underlying database files are suspended while the one or more underlying database files are being sent to the second storage node, wherein the second storage node receives the one or more underlying database files and updates a local key-value store based on the one or more received underlying database files.

Assignees

• Cohesity Inc

Inventors

• Jain Sachin
• Pallipadi Venkatesh
• Naeni Sharath Kumar

Classifications

• G06F11/1451

  by selection of backup contents · CPC title

• G06F3/0604

  Improving or facilitating administration, e.g. storage management · CPC title

• G06F11/1471

  involving logging of persistent data for recovery · CPC title

• G06F11/14

  Error detection or correction of the data by redundancy in operations (error detection or correction of the data by redundancy in hardware G06F11/16) · CPC title

• G06F2201/81

  Threshold · CPC title