Adaptive data striping and replication across multiple storage clouds for high availability and performance

Patent metadata

Field	Value
Publication number	US-9348840-B2
Application number	US-201213715241-A
Country	US
Kind code	B2
Filing date	Dec 14, 2012
Priority date	Dec 14, 2012
Publication date	May 24, 2016
Grant date	May 24, 2016

Abstract

Technologies for adaptively striping data across multiple storage clouds include receiving user constraints corresponding one or more cloud storage providers, receiving a file to be striped across the cloud storage providers, splitting the received file into file blocks, allocating each of the file blocks to a different one of the cloud storage providers as a function of the user constraints and operating conditions of each of the cloud storage providers, and sending each of the file blocks to the cloud storage provider to which each file block is allocated. In some embodiments, file blocks may be re-allocated from one cloud storage provider to another cloud storage provider as a function of changing user constraints or operating conditions. In addition, each of the file blocks may be retrieved from the cloud storage providers to re-assemble the file.

First claim

The invention claimed is:

1. A cloud resource allocation server for adaptively striping data across a plurality of storage clouds, the cloud resource allocation server comprising:
a communication hardware module to receive (i) user constraints corresponding one or more of a plurality of cloud storage providers and (ii) a file to be striped across the plurality of cloud storage providers;
a block allocation hardware module to (i) split the received file into a plurality of file blocks as a function of the user constraints and (ii) allocate each of the plurality of file blocks to a cloud storage provider of the plurality of cloud storage providers as a function of the user constraints and operating conditions of each of the plurality of cloud storage providers, each of the plurality of file blocks being allocated to a different cloud storage provider, (iii) store location metadata for each of the plurality file blocks, wherein the location metadata for each file block identifies the cloud storage provider of the plurality of cloud storage providers to which the respective file block is allocated and stored, (iv) replicate the location metadata for each of the plurality of file blocks to another cloud resource allocation server, and (v) store re-assembly metadata for re-assembling the file from each of the plurality of file blocks;
a change detection hardware module to detect a change to the operating conditions of a first cloud storage provider of the plurality of cloud storage providers storing a file block of the plurality of file blocks;
wherein the communication hardware module is further to send each of the plurality of file blocks to the cloud storage provider to which each file block is allocated to be stored; and
wherein the block allocation hardware module is further to (i) reallocate the stored file block from the first cloud storage provider to a second cloud storage provider of the plurality of cloud storage providers in response to a determination that the changed operating conditions of the first cloud storage service provider no longer satisfy the user constraints and (ii) update the location metadata for the file block reallocated from the first cloud storage provider to the second cloud storage provider.

2. The cloud resource allocation server of claim 1 , wherein the block allocation module hardware further to replicate each allocated file block to another cloud storage provider, the another cloud storage provider being different from the cloud storage provider to which each file block is allocated.

3. The cloud resource allocation server of claim 1 , wherein the user constraints comprise an availability preference, a performance level preference, and a cost preference of which each of the plurality of cloud storage providers must satisfy to be allocated one or more of the plurality of file blocks, and
wherein to allocate each of the plurality of file blocks to a cloud storage provider of the plurality of cloud storage providers as a function of the user constraints and the operating conditions of each of the plurality of cloud storage providers comprises to allocate each of the plurality of file blocks to a cloud storage provider of the plurality of cloud storage providers in response to determining that the cloud storage provider satisfies the availability preference, the performance level preference, and the cost preference.

4. The cloud resource allocation server of claim 3 , wherein to reallocate the stored file block from the first cloud storage provider to the second cloud storage provider comprises to reallocate the stored file block from the first cloud storage provider to the second cloud storage provider in response to a determination that the changed operating conditions of the first cloud storage provider no longer satisfy one or more of the availability preference, the performance level preference, and the cost preference.

5. A method for adaptively striping data across a plurality of storage clouds, the method comprising:
receiving, by a cloud resource allocation server, user constraints corresponding one or more of a plurality of cloud storage providers;
receiving, by the cloud resource allocation server, a file to be striped across the plurality of cloud storage providers;
splitting, by the cloud resource allocation server, the received file into a plurality of file blocks as a function of the user constraints;
allocating, by the cloud resource allocation server, each of the plurality of file blocks to a cloud storage provider of the plurality of cloud storage providers as a function of the user constraints and operating conditions of each of the plurality of cloud storage providers, each of the plurality of file blocks being allocated to a different cloud storage provider;
sending, by the cloud resource allocation server, each of the plurality of file blocks to the cloud storage provider to which each file block is allocated to be stored;
storing, by the cloud resource allocation server, location metadata for each of the plurality file blocks, the location metadata for each file block identifying the cloud storage provider of the plurality of cloud storage providers to which the respective file block is allocated and stored;
replicating, by the cloud resource allocation server, the location metadata for each of the plurality of file blocks to another cloud resource allocation server;
storing, by the cloud resource allocation server, re-assembly metadata for re-assembling the file from each of the plurality of file blocks;
detecting, by the cloud resource allocation server, a change to the operating conditions of a first cloud storage provider of the plurality of cloud storage providers storing a file block of the plurality of file blocks;
reallocating, by the cloud resource allocation server, the stored file block from the first cloud storage provider to a second cloud storage provider of the plurality of cloud storage providers in response to a determination that the changed operating conditions of the first cloud storage service provider no longer satisfy the user constraints; and
updating, by the cloud resource allocation server, the location metadata for the file block reallocated from the first cloud storage provider to the second cloud storage provider.

6. The method of claim 5 , further comprising:
replicating, by the cloud resource allocation server, each allocated file block to the another cloud storage provider, the another cloud storage provider being different from the cloud storage provider to which each file block is allocated.

7. The method of claim 5 , wherein the user constraints comprise an availability preference, a performance level preference, and a cost preference of which each of the plurality of cloud storage providers must satisfy to be allocated one or more of the plurality of file blocks, and
wherein allocating each of the plurality of file blocks to a cloud storage provider of the plurality of cloud storage providers as a function of the user constraints and the operating conditions of each of the plurality of cloud storage providers comprises allocating each of the plurality of file blocks to a cloud storage provider of the plurality of cloud storage providers in response to determining that the cloud storage provider satisfies the availability preference, the performance level preference, and the cost preference.

8. The method of claim 7 , wherein reallocating the stored file block from the first cloud storage provider to the second cloud storage provider comprises reallocating the stored file block from the first cloud storage provider to the second cloud storage provider in response to a determination that the changed operating conditions of the first cloud storage provider no longer

Assignees

• Intel Corp

Inventors

• Sedayao Jeffrey C
• Chahal Sudip S

Classifications

• G06F17/30212 Primary

  Physics · mapped topic

• G06F16/134

  Distributed indices · CPC title

• G06F3/0661

  Format or protocol conversion arrangements · CPC title

• G06F16/162 Primary

  Delete operations (erasing in storage systems G06F3/0652) · CPC title

• G06F16/184 Primary

  implemented as replicated file system · CPC title