Metadata synchronization in a distrubuted database

Patent metadata

Field	Value
Publication number	US-10503699-B2
Application number	US-201615137648-A
Country	US
Kind code	B2
Filing date	Apr 25, 2016
Priority date	Apr 25, 2016
Publication date	Dec 10, 2019
Grant date	Dec 10, 2019

Abstract

Disclosed herein are system, method, and computer program product embodiments for synchronizing an unsynchronized distributed database system. An embodiment operates by locking, by a second node, partitions in the second node based on first information received from, a first node responsive to execution of partition utility based on a first query received by the first node. Blocking, by the second node, a second query to a data portion of the partitions in the second node based on the first information. Locking, by the second node, second node object metadata based on second information received from the first node. Receiving, by the second node, changes to first node object metadata and applying the changes to the second node object metadata. Unlocking, by the second node, the partitions in the second node and the second node object metadata while unblocking the second query.

First claim

What is claimed is:

1. A computer implemented method, comprising:
locking, by a second node, a partition in the second node responsive to a partition utility command executed by a utility node and affecting the partition handled by the utility node, and responsive to a query received at the second node, wherein the utility node and the second node are part of a cluster;
blocking, by the second node, the query to a data portion of the partition in the second node responsive to the partition utility command;
locking, by the second node, object metadata from the second node based on a cluster-wide lock issued responsive to the partition utility command;
receiving, by the second node, changes to object metadata of the utility node and applying the changes to the object metadata of the second node; and
unlocking, by the second node, the partition and the object metadata of the second node and unblocking the query, once the changes to the object metadata of the utility node are applied to the object metadata of the second node,
wherein at least one of the locking the partition, blocking the query, locking the object metadata from the second node, receiving the changes, and unlocking the partition are performed by one or more processors.

2. The method of claim 1 , further comprising:
locking partitions in the utility node responsive to execution of the partition utility command.

3. The method of claim 1 , wherein the partition utility command comprises a merge partition, a drop partition, a move partition, a split partition, or a truncate partition.

4. The method of claim 1 , further comprising:
flushing data to persistent storage at the utility node and writing information pertaining to the partition utility command to a log file responsive to execution of the partition utility command.

5. The method of claim 1 , wherein the cluster-wide lock comprises object creation synchronization, wherein the object creation synchronization disallows creation of the second node object metadata.

6. The method of claim 1 , wherein the cluster-wide lock comprises object access synchronization, wherein the object access synchronization comprises instructions to lock in-memory structures containing the second node object metadata.

7. The method of claim 1 , further comprising:
receiving, by the second node, a flush instruction on the object metadata based on a subsequent task to the partition utility command being executed by the utility node.

8. A system, comprising:
a memory; and
at least one processor coupled to the memory and configured to:
lock, by a second node, a partition in the second node responsive to a partition utility command executed by a utility node and affecting the partition handled by the utility node, and responsive to a query received at the second node, wherein the utility node and the second node are part of a cluster;
block, by the second node, the query to a data portion of the partition in the second node responsive to the partition utility command;
lock, by the second node, object metadata of the second node based on a cluster-wide lock issued responsive to the partition utility command;
receive, by the second node, changes to object metadata of the utility node and applying the changes to the object metadata of the second node; and
unlock, by the second node, the partition and the object metadata of the second node and unblock the query, once the changes to the object metadata of the utility node are applied to the object metadata of the second node.

9. The system of claim 7 , wherein the at least one processor is further configured to:
lock partitions in the utility node responsive to execution of the partition utility command.

10. The system of claim 8 , wherein the partition utility command comprises a merge partition, a drop partition, a move partition, a split partition, or a truncate partition.

11. The system of claim 8 , wherein the at least one processor is further configured to:
flush data to persistent storage at the utility node and write information pertaining to the partition utility to a log file responsive to execution of the partition utility command.

12. The system of claim 8 , wherein the cluster-wide lock comprises object creation synchronization, wherein the object creation synchronization disallows creation of the second node object metadata.

13. The system of claim 8 , wherein the cluster-wide lock comprises object access synchronization, wherein the object access synchronization comprises instructions to lock in-memory structures containing the second node object metadata.

14. A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:
locking, by a second node, a partition in the second node responsive to a partition utility command executed by a utility node and affecting the partition handled by the utility node and responsive to a query received at the second node, wherein the utility node and the second node are part of a cluster;
blocking, by the second node, the query to a data portion of the partition in the second node responsive to the partition utility command;
locking, by the second node, object metadata of the second node based on a cluster-wide lock issued responsive to the partition utility command;
receiving, by the second node, changes to object metadata of the utility node and applying the changes to the object metadata of the second node; and
unlocking, by the second node, the partition and the object metadata of the second node and unblocking the query, once the changes to the object metadata of the utility node are applied to the object metadata of the second node.

15. The computer-readable device of claim 14 , the operations further comprising:
locking partitions in the utility node responsive to execution of the partition utility command.

16. The computer-readable device of claim 15 , wherein the execution of the partition utility command further comprises:
writing lock context information regarding the execution of the partition utility command in a log file on the utility node;
executing failover recovery on the second node in response to a failure of the execution of the partition utility command on the utility node; and
reading the lock context information from the log file and locking the partition based on the lock context information during the failover recovery.

17. The computer-readable device of claim 14 , the operations further comprising:
flushing data to persistent storage at the utility node and writing information pertaining to the partition utility to a log file responsive to execution of the partition utility.

18. The computer-readable device of claim 14 , wherein the cluster-wide lock comprises object creation synchronization, wherein the object creation synchronization disallows creation of the second node object metadata.

19. The computer-readable device of claim 14 , wherein the cluster-wide lock comprises object access synchronization, wherein the object access synchronization comprises instructions to lock in-memory structures containing object metadata on the second node.

20. The computer-readable device of claim 14 , wherein the locking the object metadata of the second node further comprises skipping flushing on the second node.

21. The computer-readable device of claim 14 , wherein the changes to the object metadata of the utility node ar

Assignees

• Sap Se

Inventors

• Gupta Gaurav Kumar
• Pal Souvik
• Pathak Amit
• Sudrik Jay

Classifications

• G06F16/182

  Distributed file systems · CPC title

• G06F16/178 Primary

  Techniques for file synchronisation in file systems · CPC title

• G06F16/27 Primary

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title