Systems and methods for distributive data storage

Patent metadata

Field	Value
Publication number	US-2020409565-A1
Application number	US-202017020096-A
Country	US
Kind code	A1
Filing date	Sep 14, 2020
Priority date	Jan 4, 2017
Publication date	Dec 31, 2020
Grant date	—

Abstract

Some embodiments can include a system. In many embodiments, a system can comprise one or more processors and one or more non-transitory storage devices storing computing instructions configured to run on the one or more processors and perform: dividing a computer file into two or more portions; calculating a respective unique identifier for each respective portion of the two or more portions; creating two or more respective computer file portion replicas of each respective portion of the two or more portions, wherein each respective computer file portion replica of the two or more respective computer file portion replicas: can be for a corresponding portion of the two or more portions; and can comprises a same one of the respective unique identifier as the corresponding portion of the two or more portions; and facilitating storage of the two or more respective computer file portion replicas on a distributive data storage cluster. Other embodiments are disclosed herein.

First claim

We claim:

1 . A system comprising:
one or more processors; and one or more non-transitory storage devices storing computing instructions configured to run on the one or more processors and perform:
dividing a computer file into two or more portions;
calculating a respective unique identifier for each respective portion of the two or more portions;
creating two or more respective computer file portion replicas of each respective portion of the two or more portions, wherein each respective computer file portion replica of the two or more respective computer file portion replicas:
is for a corresponding portion of the two or more portions; and
comprises a same one of the respective unique identifier as the corresponding portion of the two or more portions; and

facilitating storage of the two or more respective computer file portion replicas on a distributive data storage cluster.

2 . The system of claim 1 , wherein each respective unique identifier comprises a respective portion hash value, a respective label, respective metadata, or a respective metadata tag.

3 . The system of claim 1 , wherein the one or more non-transitory storage devices storing the computing instructions are further configured to run on the one or more processors and perform:
receiving, from an electronic device of a user, a request to utilize the computer file; sending a request to retrieve the computer file from the distributive data storage cluster; receiving, from the distributive data storage cluster, a set of the two or more respective computer file portion replicas, wherein the set of the two or more computer file portion replicas, when combined, comprises the computer file; and transmitting instructions allowing the electronic device of the user to utilize the computer file.

4 . The system of claim 3 , wherein the one or more non-transitory storage devices storing the computing instructions are further configured to run on the one or more processors and perform:
calculating a respective retrieved unique identifier for each respective portion of the two or more portions associated with the set of the two or more respective computer file portion replicas, as received; and determining that the respective retrieved unique identifier for each respective portion of the two or more portions associated with the set of the two or more respective computer file portion replicas, as received, matches the respective unique identifier for each respective portion of the two or more portions, thereby confirming that the two or more respective computer file portion replicas, as received, have not been corrupted.

5 . The system of claim 3 , wherein the one or more non-transitory storage devices storing the computing instructions are further configured to run on the one or more processors and perform:
merging the set of the two or more computer file portion replicas, as received, into the computer file.

6 . The system of claim 1 , wherein at least one respective computer file portion replica of the two or more respective computer file portion replicas comprises a fastest responding computer file portion replica.

7 . The system of claim 1 , wherein the computer file comprises an image file.

8 . The system of claim 1 , wherein creating the two or more respective computer file portion replicas of each respective portion of the two or more portions comprises:
replicating each respective portion of the two or more portions by a respective replication factor based, at least in part, on a number of the two or more storage nodes.

9 . The system of claim 8 , wherein the respective replication factor is determined using an equation comprising:

Q
=

N
2

+
1

,

wherein:
Q comprises the number of the two or more storage nodes; and
N comprises the respective replication factor.

10 . The system of claim 1 , wherein the respective unique identifier for each respective portion of the two or more portions is different for each respective portion of the two or more portions.

11 . A method implemented via execution of computing instructions configured to run at one or more processors and configured to be stored at non-transitory computer-readable media, the method comprising:
dividing a computer file into two or more portions; calculating a respective unique identifier for each respective portion of the two or more portions; creating two or more respective computer file portion replicas of each respective portion of the two or more portions, wherein each respective computer file portion replica of the two or more respective computer file portion replicas:
is for a corresponding portion of the two or more portions; and
comprises a same one of the respective unique identifier as the corresponding portion of the two or more portions; and
facilitating storage of the two or more respective computer file portion replicas on a distributive data storage cluster.

12 . The method of claim 11 , wherein each respective unique identifier comprises a respective portion hash value, a respective label, respective metadata, or a respective metadata tag.

13 . The method of claim 11 , wherein the one or more non-transitory storage devices storing the computing instructions are further configured to run on the one or more processors and perform:
receiving, from an electronic device of a user, a request to utilize the computer file; sending a request to retrieve the computer file from the distributive data storage cluster; receiving, from the distributive data storage cluster, a set of the two or more respective computer file portion replicas, wherein the set of the two or more computer file portion replicas, when combined, comprises the computer file; and transmitting instructions allowing the electronic device of the user to utilize the computer file.

14 . The method of claim 13 , wherein the one or more non-transitory storage devices storing the computing instructions are further configured to run on the one or more processors and perform:
calculating a respective retrieved unique identifier for each respective portion of the two or more portions associated with the set of the two or more respective computer file portion replicas, as received; and determining that the respective retrieved unique identifier for each respective portion of the two or more portions associated with the set of the two or more respective computer file portion replicas, as received, matches the respective unique identifier for each respective portion of the two or more portions, thereby confirming that the two or more respective computer file portion replicas, as received, have not been corrupted.

15 . The method of claim 13 , wherein the one or more non-transitory storage devices storing the computing instructions are further configured to run on the one or more processors and perform:
merging the set of the two or more computer file portion replicas, as received, into the computer file.

16 . The method of claim 11 , wherein at least one respective computer file portion replica of the two or more respective computer file portion replicas comprises a fastest responding computer file portion replica.

17 . The method of claim 11 , wherein the computer file comprises an imag

Assignees

• Walmart Apollo Llc

Inventors

• Sehgal Charandeep
• Bhat Vikas
• Krishnan Ganesh
• Kandaswamy Venkatesh

Classifications

• G06F16/51 Primary

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F3/065

  Replication mechanisms · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/0619

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F3/0659

  Command handling arrangements, e.g. command buffers, queues, command scheduling · CPC title