Hyper-scale P2P deduplicated storage system using a distributed ledger

Patent metadata

Field	Value
Publication number	US-12189488-B2
Application number	US-202318362801-A
Country	US
Kind code	B2
Filing date	Jul 31, 2023
Priority date	Jul 18, 2019
Publication date	Jan 7, 2025
Grant date	Jan 7, 2025

Abstract

One example method includes receiving from a node, in an HSAN that includes multiple nodes, an ADD_DATA request to add an entry to a distributed ledger of the HSAN, the request comprising a user ID that identifies the node, a hash of a data segment, and a storage location of the data segment at the node, performing a challenge-and-response process with the node to verify that the node has a copy of the data that was the subject of the entry, making a determination that a replication factor X has not been met, and adding the entry to the distributed ledger upon successful conclusion of the challenge-and-response process.

First claim

What is claimed is:

1. A method, comprising:
transmitting, by a caller, to a hyper-scale, peer-to-peer, de-duplicated storage system (HSAN) that includes multiple nodes, an ADD_DATA request to add an entry to a distributed ledger of the HSAN, the request comprising a user ID that identifies the caller, a hash of a data segment, and a storage location of the data segment at the caller;
performing, by the caller together with one or more of the nodes of the HSAN, a challenge-and-response process to verify that the caller has a copy of the data segment that was the subject of the entry;
receiving, by the caller, a unique piece of information including a salt;
re-encrypting and re-hashing, by the caller, the data segment with the salt to transform the data segment; and
providing, by the caller, the transformed data segment to the HSAN,
wherein successful conclusion of the challenge-and-response process enables the entry to be added to the distributed ledger of the HSAN so long as a replication factor X has not been met, and
wherein the replication factor X specifies a maximum number X of copies of the data segment that will be stored in the HSAN.

2. The method as recited in claim 1 , wherein the challenge-and-response process comprises receiving, by the caller, a challenge from the one or more nodes of the HSAN.

3. The method as recited in claim 2 , wherein the challenge comprises a unique piece of information usable by the caller to transform data identified in the ADD_DATA request.

4. The method as recited in claim 1 , wherein the caller is a node of the HSAN.

5. The method as recited in claim 1 , wherein an unsuccessful conclusion of the challenge-and-response process prevents the entry from being added to the distributed ledger of the HSAN.

6. The method as recited in claim 1 , wherein the transmitting comprises transmitting, by the caller, the ADD_DATA request to the multiple nodes of the HSAN.

7. The method as recited in claim 1 , wherein successful conclusion of the challenge-and-response process indicates that the nodes of the HSAN have agreed amongst themselves that the caller supplied the correct response in the challenge-and-response process.

8. The method as recited in claim 1 , wherein when conclusion of the challenge-and-response process is unsuccessful, the caller receives a notification that the replication factor X has already been met and no new entry to the ledger will be made.

9. The method as recited in claim 1 , wherein distributed ledger of the HSAN comprises a public distributed ledger.

10. The method as recited in claim 1 , wherein the caller comprises a copy of the distributed ledger of the HSAN.

11. A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
transmitting, by a caller, to a hyper-scale, peer-to-peer, de-duplicated storage system (HSAN) that includes multiple nodes, an ADD_DATA request to add an entry to a distributed ledger of the HSAN, the request comprising a user ID that identifies the caller, a hash of a data segment, and a storage location of the data segment at the caller;
performing, by the caller together with one or more of the nodes of the HSAN, a challenge-and-response process to verify that the caller has a copy of the data segment that was the subject of the entry;
receiving, by the caller, a unique piece of information including a salt;
re-encrypting and re-hashing, by the caller, the data segment with the salt to transform the data segment; and
providing, by the caller, the transformed data segment to the HSAN,
wherein successful conclusion of the challenge-and-response process enables the entry to be added to the distributed ledger of the HSAN so long as a replication factor X has not been met, wherein the replication factor X specifies a maximum number X of copies of the data segment that will be stored in the HSAN.

12. The non-transitory storage medium as recited in claim 11 , wherein the challenge-and-response process comprises receiving, by the caller, a challenge from the one or more nodes of the HSAN.

13. The non-transitory storage medium as recited in claim 12 , wherein the challenge comprises a unique piece of information usable by the caller to transform data identified in the ADD_DATA request.

14. The non-transitory storage medium as recited in claim 11 , wherein the caller is a node of the HSAN.

15. The non-transitory storage medium as recited in claim 11 , wherein an unsuccessful conclusion of the challenge-and-response process prevents the entry from being added to the distributed ledger of the HSAN.

16. The non-transitory storage medium as recited in claim 11 , wherein the transmitting comprises transmitting, by the caller, the ADD_DATA request to the multiple nodes of the HSAN.

17. The non-transitory storage medium as recited in claim 11 , wherein successful conclusion of the challenge-and-response process indicates that the nodes of the HSAN have agreed amongst themselves that the caller supplied the correct response in the challenge-and-response process.

18. The non-transitory storage medium as recited in claim 11 , wherein when conclusion of the challenge-and-response process is unsuccessful, the caller receives a notification that the replication factor X has already been met and no new entry to the ledger will be made.

19. The non-transitory storage medium as recited in claim 11 , wherein distributed ledger of the HSAN comprises a public distributed ledger.

20. The non-transitory storage medium as recited in claim 11 , wherein the caller comprises a copy of the distributed ledger of the HSAN.

Assignees

• Emc Ip Holding Co Llc

Inventors

• Murti Arun
• Lei Joey C
• Brenner Adam E
• Malamut Mark D

Classifications

• H04L9/50

  using hash chains, e.g. blockchains or hash trees · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• H04L9/3271

  using challenge-response · CPC title

• H04L9/3236

  using cryptographic hash functions · CPC title

• H04L9/3213

  using tickets or tokens, e.g. Kerberos (network architectures or network communication protocols for entities authentication using tickets in a packet data network H04L63/0807) · CPC title