Transitioning leadership in a cluster of nodes

Patent metadata

Field	Value
Publication number	US-12067131-B2
Application number	US-202217902413-A
Country	US
Kind code	B2
Filing date	Sep 2, 2022
Priority date	Apr 24, 2018
Publication date	Aug 20, 2024
Grant date	Aug 20, 2024

Abstract

Transitioning leadership in a cluster of nodes, including: initiating, by two or more nodes among a cluster of nodes, a leadership transition, wherein: a first node transmits a first secret key identifier to each of the other nodes in the cluster of nodes; and a second node transmits a second secret key identifier to each of the other nodes in the cluster of nodes; updating, by each node and based at least in part on a resolution policy, the current secret key identifier to be the second secret key identifier instead of the first secret key identifier; and transitioning, based at least in part on the second secret key identifier being selected to be the current secret key identifier, the second node to be a leader node of the cluster of nodes.

First claim

What is claimed is:

1. A method comprising:
initiating, by two or more nodes among a cluster of nodes, a leadership transition from a current leader node with a current secret key identifier to a new leader node among the cluster of nodes, wherein:
a first node transmits a first secret key identifier to each of the other nodes in the cluster of nodes; and
a second node transmits a second secret key identifier to each of the other nodes in the cluster of nodes, the first and second nodes distinct from the current leader node;

updating, by each node and based at least in part on a resolution policy, the current secret key identifier to be the second secret key identifier instead of the first secret key identifier; and
transitioning, based at least in part on the second secret key identifier being selected to be the current secret key identifier, the second node to be the new leader node of the cluster of nodes.

2. The method of claim 1 , wherein the nodes are included in a deduplication cluster that comprises multiple servers within an intermediate computing system between one or more client devices and a backend cloud storage service.

3. The method of claim 2 , wherein a data store on the intermediate computing system uses a consistent data storage model, and wherein the data store for the backend cloud storage service uses an eventually consistent data storage model.

4. The method of claim 1 , wherein the method further comprises:
receiving, via an application program interface of a front-end process, a stream of data;
splitting the stream of data into blocks of data;
hashing the blocks of data;
determining, whether a hash value for a block of data from among the blocks of data, is a duplicate of a hash value for a stored block of data;
responsive to the hash value for the block of data not matching the hash value for the stored block of data, routing the block of data to a process from among the cluster of nodes; and
repeating, for each given hash for a given block of data of the blocks of data:
determining whether the given hash is a duplicate of some hash value for some stored block of data, and
responsive to the given hash value for the given block of data not matching some hash value for some stored block of data, routing the given block of data to a process from among the cluster of nodes.

5. The method of claim 4 , further comprising:
distributing, to different ones of the cluster of nodes, the hashed blocks of data that are not duplicates of stored data.

6. The method of claim 5 , further comprising:
sending, from a process that has received a hashed block of data to a remote data store, one or more portions of the block of data, wherein the one or more portions of the block of data correspond to one or more transactions.

7. The method of claim 6 , wherein the one or more transactions are recorded within a transaction log for the process, and wherein each process among the cluster of nodes generates a transaction log corresponding to data sent to the remote data store.

8. The method of claim 7 , wherein the front-end process receives the stream of bytes of data from a client device via a communication interface that is compatible with a communication interface provided by the remote data store.

9. The method of claim 7 , wherein the remote data store is an object store provided by a cloud services provider.

10. The method of claim 1 , wherein at least one process among the cluster of nodes operates in parallel with at least one other process among the cluster of nodes.

11. A system comprising two or more nodes, wherein the system is a storage system and the two or more nodes are storage nodes and the system is configured to perform the steps of:
initiating, by two or more nodes among a cluster of nodes, a leadership transition from a current leader node with a current secret key identifier to a new leader node among the cluster of nodes, wherein:
a first node transmits a first secret key identifier to each of the other nodes in the cluster of nodes; and
a second node transmits a second secret key identifier to each of the other nodes in the cluster of nodes, the first and second nodes distinct from the current leader node;

updating, by each node and based at least in part on a resolution policy, the current secret key identifier to be the second secret key identifier instead of the first secret key identifier; and
transitioning, based at least in part on the second secret key identifier being selected to be the current secret key identifier, the second node to be the new leader node of the cluster of nodes.

12. The system of claim 11 , wherein the system comprises multiple servers within an intermediate computing system between one or more client devices and a backend cloud storage service.

13. The system of claim 12 , wherein a data store on the intermediate computing system uses a consistent data storage model, and wherein the data store for the backend cloud storage service uses an eventually consistent data storage model.

14. The system of claim 11 , wherein the system is further configured to perform the steps of:
receiving, via an application program interface of a front-end process, a stream of data;
splitting the stream of data into blocks of data;
hashing the blocks of data;
determining, whether a hash value for a block of data from among the blocks of data, is a duplicate of a hash value for a stored block of data;
responsive to the hash value for the block of data not matching the hash value for the stored block of data, routing the block of data to a process from among the cluster of nodes; and
repeating, for each given hash for a given block of data of the blocks of data:
determining whether the given hash is a duplicate of some hash value for some stored block of data, and
responsive to the given hash value for the given block of data not matching some hash value for some stored block of data, routing the given block of data to a process from among the cluster of nodes.

15. The system of claim 14 , wherein the system is further configured to perform the step of:
distributing, to different ones of the cluster of nodes, the hashed blocks of data that are not duplicates of stored data.

16. The system of claim 15 , wherein the system is further configured to perform the step of:
sending, from a process that has received a hashed block of data to a remote data store, one or more portions of the block of data, wherein the one or more portions of the block of data correspond to one or more transactions.

17. The system of claim 16 , wherein the one or more transactions are recorded within a transaction log for the process, and wherein each process among the cluster of nodes generates a transaction log corresponding to data sent to the remote data store.

18. The system of claim 17 , wherein the front-end process receives the stream of bytes of data from a client device via a communication interface that is compatible with a communication interface provided by the remote data store.

19. The system of claim 17 , wherein the remote data store is an object store provided by a cloud services provider.

20. A computer program product disposed upon a non-transitory computer readable medium, the computer program product comprising computer program instructions that, when executed, cause a computer to carry out the steps of:
initiating, by two or more nodes among a cluster of nodes, a leadership transition from a current leader node with a current secret key identifier t

Assignees

• Pure Storage Inc

Inventors

• Juch Ian
• Power Tyler

Classifications

• H04L9/0643

  Hash functions, e.g. MD5, SHA, HMAC or f9 MAC · CPC title

• G06F16/2365

  Ensuring data consistency and integrity · CPC title

• G06F16/2379

  Updates performed during online database operations; commit processing · CPC title

• G06F16/215

  Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors · CPC title

• H04L9/0891

  Revocation or update of secret information, e.g. encryption key update or rekeying · CPC title