Failure recovery in a scaleout system using vector clock

Patent metadata

Field	Value
Publication number	US-2025086068-A1
Application number	US-202318464726-A
Country	US
Kind code	A1
Filing date	Sep 11, 2023
Priority date	Sep 11, 2023
Publication date	Mar 13, 2025
Grant date	—

Abstract

Determining a failure recovery line in a distributed scaleout computing system. Each node or process of a distributed system has or is associated with a vector clock that includes a logical clock for each node in the distributed system. When failure is detected, a recovery operation may be performed using the vector clock. After the recovery operation, the vector clock is updated such that the failure recovery line is available in the computing system.

First claim

What is claimed is:

1 . A method comprising:
associating a vector clock with each node in a distributed scaleout system; when an event occurs at a node, updating a vector clock associated with the node; determining a failure in the distributed scaleout system; performing a rollback operation using the vector clocks, wherein the vector clocks identify a failure recovery line for recovering the distributed scaleout system from the failure; and performing a recovery operation based on the failure recovery line.

2 . The method of claim 1 , wherein each of the vector clocks includes a home logical clock for a corresponding node and one or more away logical clocks for other nodes in the distributed Scaleout system.

3 . The method of claim 1 , wherein the event is an internal event, further comprising updating the home logical clock in the vector clock of the node that experienced the internal event.

4 . The method of claim 1 , wherein the event is an external event, further comprising updating a home logical clock in the vector clock of the node and updating the vector clock of the node based using a vector clock of a sending node that sent the event to the node.

5 . The method of claim 1 , further comprising performing a cascaded rollback operation using the vector clocks.

6 . The method of claim 5 , wherein the recovery operation includes recovering from a snapshot and replaying logs, wherein the logs store a chronological history of events in the distributed scaleout system.

7 . The method of claim 6 , further comprising updating the vector clocks such that the vector clocks include a failure recovery line that accounts for the recovery operation.

8 . The method of claim 7 , wherein some entries in the vector clocks are deleted and replaced with new entries.

9 . The method of claim 1 , further comprising scaling the distributed computing system, wherein a vector clock of a new node added to the distributed computing system is initialized to zeros.

10 . The method of claim 1 , further comprising moving multiple nodes forward from a recovery line after using the failure recovery line to identify the recovery line.

11 . A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
associating a vector clock with each node in a distributed scaleout system; when an event occurs at a node, updating a vector clock associated with the node; determining a failure in the distributed scaleout system; performing a rollback operation using the vector clocks, wherein the vector clocks identify a failure recovery line for recovering the distributed scaleout system from the failure; and performing a recovery operation based on the failure recovery line.

12 . The non-transitory storage medium of claim 11 , wherein each of the vector clocks includes a home logical clock for a corresponding node and one or more away logical clocks for other nodes in the distributed Scaleout system.

13 . The non-transitory storage medium of claim 11 , wherein the event is an internal event, further comprising updating the home logical clock in the vector clock of the node that experienced the internal event.

14 . The non-transitory storage medium of claim 11 , wherein the event is an external event, further comprising updating a home logical clock in the vector clock of the node and updating the vector clock of the node based using a vector clock of a sending node that sent the event to the node.

15 . The non-transitory storage medium of claim 11 , further comprising performing a cascaded rollback operation using the vector clocks.

16 . The non-transitory storage medium of claim 15 , wherein the recovery operation includes recovering from a snapshot and replaying logs, wherein the logs store a chronological history of events in the distributed scaleout system.

17 . The non-transitory storage medium of claim 16 , further comprising updating the vector clocks such that the vector clocks include a failure recovery line that accounts for the recovery operation.

18 . The non-transitory storage medium of claim 17 , wherein some entries in the vector clocks are deleted and replaced with new entries.

19 . The non-transitory storage medium of claim 11 , further comprising scaling the distributed computing system, wherein a vector clock of a new node added to the distributed computing system is initialized to zeros.

20 . The non-transitory storage medium of claim 11 , further comprising moving multiple nodes forward from a recovery line after using the failure recovery line to identify the recovery line.

Assignees

• Dell Products Lp

Inventors

• Desai Keyur B

Classifications

• G06F11/1461

  Backup scheduling policy · CPC title

• G06F11/0709 Primary

  in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems · CPC title

• G06F11/1469 Primary

  Backup restoration techniques · CPC title

• G06F11/1471

  involving logging of persistent data for recovery · CPC title

• G06F11/08

  Error detection or correction by redundancy in data representation, e.g. by using checking codes · CPC title