Storing data for machine learning and artificial intelligence applications in a decentralized storage network

Patent metadata

Field	Value
Publication number	US-12216927-B2
Application number	US-202117466198-A
Country	US
Kind code	B2
Filing date	Sep 3, 2021
Priority date	Mar 9, 2018
Publication date	Feb 4, 2025
Grant date	Feb 4, 2025

Abstract

Storing data for machine learning and artificial intelligence applications in a decentralized storage network, including: identifying a plurality of decentralized storage networks that a storage system can utilize for storing data, each of the plurality of decentralized storage networks comprising a collection of network connected computers operating as cooperative participants without employing dedicated servers for the storage of data; selecting, based characteristics of each decentralized storage network, one or more decentralized storage networks for storing the data; and initiating storage of the data on the selected one of more decentralized storage networks.

First claim

What is claimed is:

1. A method comprising:
selecting, based on configurations of one or more decentralized storage networks from a plurality of decentralized storage networks that are aligned with requirements associated with storing data, one or more decentralized storage networks for storing the data from the plurality of decentralized storage networks that are utilized for storing data, wherein the one or more decentralized storage networks of the plurality of decentralized storage networks comprise a collection of network connected computers operating as cooperative participants without employing dedicated servers for the storage of data;
initiating storage of the data on the selected one of more decentralized storage networks, and
sharing, with one or more other decentralized storage networks, information describing a storage system that accesses the data from the one or more decentralized storage networks, wherein the data is restored using the one or more other decentralized storage networks based on the shared information.

2. The method of claim 1 further comprising reducing, prior to initiating storage of the data on the selected one of more decentralized storage networks, a size of the data using one or more data reduction techniques.

3. The method of claim 1 wherein initiating storage of the data on the selected one of more decentralized storage networks further comprises including, in a blockchain, a contract describing a relationship agreement between the storage system and a particular decentralized storage network.

4. The method of claim 1 wherein initiating storage of the data on the selected one of more decentralized storage networks further comprises including, in a blockchain, the data.

5. The method of claim 1 further comprising:
receiving a request to take a snapshot of a dataset stored within the storage system;
creating the snapshot of the dataset stored within the storage system; and
wherein initiating storage of the data on the selected one of more decentralized storage networks further comprises initiating storage of the snapshot on the selected one of more decentralized storage networks.

6. The method of claim 1 further comprising mirroring, between the plurality of decentralized storage networks, the data.

7. The method of claim 1 further comprising:
detecting that a dataset stored within the storage system should be replicated to a second storage system;
receiving a request to modify the dataset;
modifying the dataset stored within the storage system; and
wherein initiating storage of the data on the selected one of more decentralized storage networks further comprises initiating storage of the modified dataset on the selected one of more decentralized storage networks.

8. The method of claim 1 wherein the data is utilized by an artificial intelligence application.

9. The method of claim 1 wherein the data is utilized by a machine learning application.

10. An apparatus comprising a computer processor and a computer memory operatively coupled to the computer processor, the computer memory having disposed within its computer program instructions that, when executed by the computer processor, cause the apparatus to:
select, based on configurations of one or more decentralized storage networks from a plurality of decentralized storage networks that are aligned with requirements associated with storing data, one or more decentralized storage networks for storing the data from the plurality of decentralized storage networks that are utilized for storing data, wherein the one or more decentralized storage networks of the plurality of decentralized storage networks comprise a collection of network connected computers operating as cooperative participants without employing dedicated servers for the storage of data;
initiate storage of the data on the selected one of more decentralized storage networks, and
share, with one or more other decentralized storage networks, information describing a storage system that accesses the data from the one or more decentralized storage networks, wherein the data is restored using the one or more other decentralized storage networks based on the shared information.

11. The apparatus of claim 10 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to reduce, prior to initiating storage of the data on the selected one of more decentralized storage networks, a size of the data using one or more data reduction techniques.

12. The apparatus of claim 10 wherein initiating storage of the data on the selected one of more decentralized storage networks further comprises including, in a blockchain, a contract describing a relationship agreement between the storage system and a particular decentralized storage network.

13. The apparatus of claim 10 wherein initiating storage of the data on the selected one of more decentralized storage networks further comprises including, in a blockchain, the data.

14. The apparatus of claim 10 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to:
receive a request to take a snapshot of a dataset stored within the storage system;
create the snapshot of the dataset stored within the storage system; and
wherein initiating storage of the data on the selected one of more decentralized storage networks further comprises initiating storage of the snapshot on the selected one of more decentralized storage networks.

15. The apparatus of claim 10 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to mirror, between the plurality of decentralized storage networks, the data.

16. The apparatus of claim 10 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to:
detect that a dataset stored within the storage system should be replicated to a second storage system;
receive a request to modify the dataset;
modify the dataset stored within the storage system; and
wherein initiating storage of the data on the selected one of more decentralized storage networks further comprises initiating storage of the modified dataset on the selected one of more decentralized storage networks.

17. A non-transitory computer readable storage medium storing instructions which, when executed, cause a processor to:
select, based on configurations of one or more decentralized storage networks from a plurality of decentralized storage networks that are aligned with requirements associated with storing data, one or more decentralized storage networks for storing the data from the plurality of decentralized storage networks that are utilized for storing data, wherein the one or more decentralized storage networks of the plurality of decentralized storage networks comprise a collection of network connected computers operating as cooperative participants without employing dedicated servers for the storage of data;
initiate storage of the data on the selected one of more decentralized storage networks, and
share, with one or more other decentralized storage networks, information describing a storage system that accesses the data from the one or more decentralized storage networks, wherein the data is restored using the one or more other decentralized storage networks based on the shared information.

18. The non-transitory computer readable storage medium of claim 17 , the processor further configured to reduce, prior to initiating st

Assignees

• Pure Storage Inc

Inventors

• Richardson Michael

Classifications

• H04L67/1095

  Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes · CPC title

• H04L67/1097 Primary

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F2201/84

  Using snapshots, i.e. a logical point-in-time copy of the data · CPC title

• G06F11/1076

  Parity data used in redundant arrays of independent storages, e.g. in RAID systems · CPC title

• G06F3/0653

  Monitoring storage devices or systems · CPC title