Systems and methods for distributive data storage

Patent metadata

Field	Value
Publication number	US-10209901-B2
Application number	US-201715398034-A
Country	US
Kind code	B2
Filing date	Jan 4, 2017
Priority date	Jan 4, 2017
Publication date	Feb 19, 2019
Grant date	Feb 19, 2019

Abstract

In some embodiments, a method can comprise dividing an image into two or more portions and calculating a portion hash value for each portion of the two or more portions, each portion hash value for each portion of the two or more portions is different than an other portion hash value for other portions of the two or more portions. The method can further comprise creating one or more image portion replicas of each portion of the two or more portions by replicating each portion of the two or more portions by a replication factor, the one or more image portion replicas comprising the same portion hash value as each portion of the two or more portions. The method can further comprise sending the one or more image portion replicas to a distributive data storage cluster, the distributive data storage cluster comprising two or more storage nodes, and storing each of the one or more image portion replicas in the two or more storage nodes. Other embodiments of related methods and systems are also provided.

First claim

What is claimed is:

1. A system for more efficient upload, storage, and retrieval of images, the system comprising:
one or more processing modules; and
one or more non-transitory storage modules storing computing instructions configured to run on the one or more processing modules and perform acts of:
dividing an image into two or more portions;
calculating a portion hash value for each portion of the two or more portions, wherein each portion hash value for each portion of the two or more portions is different than other portion hash values for other portions of the two or more portions;
creating one or more image portion replicas of each portion of the two or more portions by replicating each portion of the two or more portions by a replication factor, the one or more image portion replicas comprising a same portion hash value as each corresponding portion of the two or more portions;
sending the one or more image portion replicas to a distributive data storage cluster, the distributive data storage cluster comprising two or more storage nodes;
storing each of the one or more image portion replicas in the two or more storage nodes, thereby resulting in the more efficient upload, storage, and retrieval of the image;
receiving, from an electronic device of a user, a request to view the image;
sending a request to retrieve the image from the distributive data storage cluster;
retrieving the image from the distributive data storage cluster by retrieving a set of the one or more image portion replicas, wherein the set of the one or more image portion replicas, when combined, comprises the image, thereby resulting in the more efficient retrieval of the image; and
transmitting instructions to display, on the electronic device of the user, the image.

2. The system of claim 1 , wherein:
dividing the image into the two or more portions comprises determining that an image size of the image is greater than 2 megabytes.

3. The system of claim 2 , wherein:
a portion size of each portion of the two or more portions is less than 2 megabytes.

4. The system of claim 1 , wherein the one or more non-transitory storage modules storing the computing instructions are configured to run on the one or more processing modules and further perform acts of:
calculating a retrieved portion hash value for each portion of the two or more portions associated with the set of the one or more image portion replicas retrieved; and
determining that the retrieved portion hash value for each portion of the two or more portions associated with the set of the one or more image portion replicas retrieved matches the portion hash value for each portion of the two or more portions, thereby confirming that the one or more image portion replicas retrieved have not been corrupted.

5. The system of claim 1 , wherein the one or more non-transitory storage modules storing the computing instructions are configured to run on the one or more processing modules and further perform an act of:
merging the set of the one or more image portion replicas into the image.

6. The system of claim 1 , wherein:
each of the one or more image portion replicas of the set of the one or more image portion replicas comprise a fastest responding image portion replica.

7. The system of claim 1 , wherein:
the distributive data storage cluster comprises three storage nodes.

8. The system of claim 1 , wherein:
the replication factor is based at least in part on a number of the two or more storage nodes in the distributive data storage cluster.

9. A method for more efficient upload, storage, and retrieval of images, the method comprising:
dividing an image into two or more portions;
calculating a portion hash value for each portion of the two or more portions, wherein each portion hash value for each portion of the two or more portions is different than other portion hash values for other portions of the two or more portions;
creating one or more image portion replicas of each portion of the two or more portions by replicating each portion of the two or more portions by a replication factor, the one or more image portion replicas comprising a same portion hash value as each corresponding portion of the two or more portions;
sending the one or more image portion replicas to a distributive data storage cluster, the distributive data storage cluster comprising two or more storage nodes;
storing each of the one or more image portion replicas in the two or more storage nodes, thereby resulting in the more efficient upload, storage, and retrieval of the image;
receiving, from an electronic device of a user, a request to view the image;
sending a request to retrieve the image from the distributive data storage cluster;
retrieving the image from the distributive data storage cluster by retrieving a set of the one or more image portion replicas, wherein the set of the one or more image portion replicas, when combined, comprises the image, thereby resulting in the more efficient retrieval of the image; and
transmitting instructions to display, on the electronic device of the user, the image.

10. The method of claim 9 , wherein:
dividing the image into the two or more portions comprises determining that an image size of the image is greater than 2 megabytes.

11. The method of claim 9 , further comprising:
calculating a retrieved portion hash value for each portion of the two or more portions associated with the set of the one or more image portion replicas retrieved; and
determining that the retrieved portion hash value for each portion of the two or more portions associated with the set of the one or more image portion replicas retrieved matches the portion hash value for each portion of the two or more portions, thereby confirming that the one or more image portion replicas retrieved have not been corrupted.

12. The method of claim 9 , further comprising:
merging the set of the one or more image portion replicas into the image.

13. The method of claim 9 , wherein:
each of the one or more image portion replicas of the set of the one or more image portion replicas comprise a fastest responding image portion replica.

14. The method of claim 9 , wherein:
the distributive data storage cluster comprises three storage nodes.

15. The method of claim 9 , wherein:
the replication factor is based at least in part on a number of the two or more storage nodes in the distributive data storage cluster.

16. A method for more efficient upload, storage, and retrieval of images, the method comprising:
dividing an image into at least a first portion and a second portion;
calculating a first portion hash value for the first portion and a second portion hash value for the second portion, wherein the first portion hash value and the second portion hash value are different from each other;
creating one or more first image portion replicas of the first portion and one or more second image portion replicas of the second portion, wherein:
a number of the one or more first image portion replicas and the one or more second image portion replicas are based at least in part on a replication factor;
the one or more first image portion replicas comprise the first portion hash value; and
the one or more second image portion replicas comprise the second portion hash value;

sending the one or more first image portion replicas and the one or more second image portion replicas to a distributive data storage cluster, the distributive data storage cluster comprising two or more storage nodes;
storing each of the one or more first image portion replicas

Assignees

• Wal Mart Stores Inc
• Walmart Apollo Llc

Inventors

• Sehgal Charandeep
• Bhat Vikas
• Krishnan Ganesh
• Kandaswamy Venkatesh

Classifications

• G06F3/0613 Primary

  in relation to throughput · CPC title

• G06F3/0659

  Command handling arrangements, e.g. command buffers, queues, command scheduling · CPC title

• G06F3/065

  Replication mechanisms · CPC title

• G06F3/0619

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title