Object storage and access management systems and methods

Patent metadata

Field	Value
Publication number	US-11662938-B2
Application number	US-202117316262-A
Country	US
Kind code	B2
Filing date	May 10, 2021
Priority date	May 11, 2020
Publication date	May 30, 2023
Grant date	May 30, 2023

Abstract

A geographically distributed erasure coding system includes multiple computer readable, non-transitory storage memories capable of storing a digital dataset including multiple object blocks, where each storage memory is configured to store one or more of the object blocks of the dataset according to an erasure coding policy. The system includes one or more processors configured to implement the erasure coding policy by distributing the multiple object blocks of the dataset to the multiple storage memories according to distribution criteria of the erasure coding policy, and the distribution criteria include at least one status parameter associated with each storage memory. The multiple storage memories are geographically distributed at different locations from one another.

First claim

What is claimed is:

1. A geographically distributed erasure coding system, the system comprising:
multiple computer readable, non-transitory storage memories capable of storing a digital dataset including multiple object blocks, each storage memory configured to store one or more of the multiple object blocks of the dataset according to an erasure coding policy; and
one or more processors, the one or more processors configured to implement the erasure coding policy by distributing the multiple object blocks of the dataset to the multiple storage memories according to distribution criteria of the erasure coding policy, the distribution criteria including at least one status parameter associated with each storage memory;
wherein the multiple storage memories are geographically distributed at different locations from one another,
wherein each geographic location is assigned a different one of multiple hash value addresses, and each of the multiple storage memories includes the hash value address corresponding to the geographic location of the storage memory, and
wherein the one or more processors are configured to generate an index of the multiple storage memories according to the hash value addresses, to distribute a first portion of the multiple object blocks to a closest one of the multiple storage memories having a hash value address in the index, and to distribute a second portion of the multiple object blocks sequentially to a next closest one of the multiple storage memories having a hash value address in the index.

2. The system of claim 1 , wherein the at least one status parameter includes, for each storage memory, at least one of a current health status of a network connected to the storage memory, a latency to transmit data to the storage memory, and an available bandwidth to transmit data to the storage memory.

3. The system of claim 1 , wherein the at least one status parameter includes, for each storage memory, at least one of a specific geographic location of the storage memory, a distance between the specific geographic location of the storage memory and a source of the dataset, and a risk of data loss at the specific geographic location of the storage memory.

4. The system of claim 1 , wherein the at least one status parameter includes, for each storage memory, at least one of a specified data protection level of the storage memory and a legal privacy or digital rights management requirement of the dataset.

5. The system of claim 1 , wherein the at least one status parameter includes, for each storage memory, at least one of a cost of utilizing the storage memory, a capacity of the storage memory, and a current availability of the storage memory.

6. The system of claim 1 , wherein the one or more processors are configured to redistribute the multiple object blocks of the dataset amongst the multiple storage memories, a specified time period after initial distribution of the multiple object blocks, and the redistributed multiple object blocks are allocated amongst the multiple storage memories in a different configuration than the allocation of the multiple object blocks in the initial distribution.

7. The system of claim 6 , wherein the one or more processors are configured to redistribute the multiple object blocks by moving the multiple object blocks to locations in the multiple storage memories having a lower cost of storage than the initial distribution.

8. The system of claim 1 , wherein a data size of one of the multiple object blocks is different than a data size of another one of the multiple object blocks.

9. The system of claim 1 , wherein the multiple storage memories are arranged in one of a tree storage topology, a linear storage topology, a star storage topology and a circle storage topology.

10. The system of claim 1 , wherein a subset of the multiple object blocks includes parity data and/or error checking data.

11. The system of claim 1 , wherein:
the dataset includes metadata indicative of one or more properties of the dataset; and
the one or more processors are configured to operate a state machine to interpret the metadata and generate the erasure coding policy according to the interpreted metadata.

12. The system of claim 1 , wherein the dataset includes at least one of health data, bioinformatics data, video data, and genomic data.

13. The system of claim 1 , wherein geographic locations of each of the multiple storage memories are indexed in a hash space, and the one or more processors are configured to torrent data to hash value addresses in a same hash space.

14. The system of claim 1 , wherein each of the multiple storage memories is configured to monitor the at least one status parameter associated with the storage memory, and transmit the monitored at least one status parameter to the one or more processors to implement the erasure coding policy.

15. The system of claim 1 , wherein the erasure coding policy includes a Reed-Solomon erasure coding implementation.

16. The system of claim 1 , wherein the distribution criteria includes at least two status parameters associated with each storage memory.

17. A computer-implemented method of managing a geographically distributed erasure coding system, the system including multiple computer readable, non-transitory storage memories for storing a digital dataset including multiple object blocks, each storage memory configured to store one or more object blocks of the dataset according to an erasure coding policy, the method comprising:
implementing, via one or more processors, the erasure coding policy by distributing the multiple object blocks of the dataset to the multiple storage memories according to distribution criteria of the erasure coding policy, wherein:
the distribution criteria includes at least one status parameter associated with each storage memory;
the multiple storage memories are geographically distributed at different locations from one another; and
each geographic location is assigned a different one of multiple hash value addresses, a location attribute of each of the storage memories includes an S2 cell identifier associated with the geographic location of the storage memory, and distributing the multiple object blocks includes distributing a first portion of the multiple object blocks to a closest one of the multiple storage memories having a hash value address in the index, and distributing a second portion of the multiple object blocks sequentially to a next closest one of the multiple storage memories having a hash value address in the index.

18. The method of claim 17 , further comprising redistributing the multiple object blocks of the dataset amongst the multiple storage memories, a specified time period after initial distribution of the multiple object blocks, wherein the redistributed multiple object blocks are allocated amongst the multiple storage memories in a different configuration than the allocation of the multiple object blocks in the initial distribution.

19. The method of claim 18 , wherein redistributing the multiple object blocks includes moving the multiple object blocks to locations in the multiple storage memories having a lower cost of storage than the initial distribution.

20. A non-transitory computer readable medium including computer-executable instructions, wherein the computer-executable instructions are executable by a processor to:
implement an erasure coding policy by distributing multiple object blocks of a digital dataset to multiple storage memories according to distribution crit

Assignees

• Nantcell Inc

Inventors

• Bassett Aaron Randolph

Classifications

• G06F21/6254

  by anonymising data, e.g. decorrelating personal data from the owner's identification · CPC title

• G06F21/64

  Protecting data integrity, e.g. using checksums, certificates or signatures · CPC title

• G06F3/064

  Management of blocks · CPC title

• G06F3/0652 Primary

  Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket · CPC title

• H03M13/1515

  Reed-Solomon codes · CPC title