Object-backed block-based distributed storage

Patent metadata

Field	Value
Publication number	US-11775569-B2
Application number	US-201715583921-A
Country	US
Kind code	B2
Filing date	May 1, 2017
Priority date	May 23, 2014
Publication date	Oct 3, 2023
Grant date	Oct 3, 2023

Abstract

An object-backed block-based storage system may store data blocks in a block cache in a non-relational database and data block objects in an object data store. Access requests for a particular data block may be received at a storage protocol target that are formatted according to a network-based storage protocol. Translated access requests formatted according to respective programmatic interfaces may be generated and sent to the non-relational database or to the object data store in order to access the particular data block. A response indicating success or failure of the access request may be sent.

First claim

What is claimed is:

1. A method, comprising:
performing, by one or more computing devices:
receiving, from a client, a read request for a particular data block in virtual block storage, wherein the read request is formatted according to a network-based storage protocol, wherein the virtual block storage is implemented using an object data store storing a plurality of data block objects for the virtual block storage and a non-relational database storing a block cache that includes a plurality of cached block entries for the virtual block storage; and
in response to receiving the read request:
generating a translated read request, comprising formatting the read request according to a programmatic interface for the non-relational database and sending, over a network, the translated read request to the non-relational database to get a particular cached block entry corresponding to the particular data block;
generating another translated read request comprising formatting the read request according to a programmatic interface for the object data store and sending, over the network, the other translated read request to the object data store to get a particular data block object corresponding to the particular data block; and
sending to the client a response to the read request formatted according to the network-based storage protocol, wherein the response includes a first one received of either the particular cached data block entry from the non-relational database or the particular data block object from the object data store.

2. The method of claim 1 , further comprising:
identifying one or more of the plurality of cached data block entries to move to a respective one or more data block objects in the plurality of data block objects at the object data store;
selecting at least one of the identified cached data block entries according to a cache management scheme;
sending the at least one cached data block entry to update the respective data block object for the at least one cached data block entry; and
in response to determining that the respective data block object was updated, deleting the at least one cached data block entry from the non-relational database.

3. The method of claim 2 , wherein the cache management scheme comprises user-defined size for the block cache, wherein the at least one identified cached block entry is selected in order to enforce the user-defined size of the block cache.

4. The method of claim 1 , where said receiving the read request and said sending the translated read requests are performed in such a way as to be Portable Operating System Interface (POSIX) compliant.

5. The method of claim 1 , further comprising:
receiving an allocation request for the virtual block storage from a storage client;
in response to receiving the allocation request:
allocating the block cache in the non-relational database and the plurality of data objects in the object data store;
provisioning a storage protocol target for receiving access requests for the virtual block storage; and
sending an identifier for the storage protocol target to the storage client such that the storage client may direct access requests formatted according to the network-based storage protocol to the storage protocol target.

6. A system, comprising:
a plurality of compute nodes, wherein individual ones of the compute nodes comprise a respective processor and respective memory, the compute nodes configured to implement an object-backed block-based storage service of a provider network that provides virtual block storage, comprising:
a multi-tenant object data storage service of the provider network, configured to store a plurality of data block objects for the virtual block storage;
a multi-tenant non-relational database service of the provider network, configured to store a plurality of cached block entries in a block cache for the virtual block storage; and
one or more storage protocol targets, each configured to:
receive, from a client, an access request for a particular data block, wherein the access request is formatted according to a network-based storage protocol;
in response to the receipt of the access request, generate translated access requests comprising:
a first translated access request for the non-relational database service in order to access a particular cached block entry corresponding to the particular data block; and
a second translated access request for the object data storage service in order to access a particular data block object corresponding to the particular data block;
wherein each of the translated access requests is formatted according to a respective programmatic interface for a respective one of the non-relational database service or the object data storage service;

send the first translated access request to the non-relational database service and the second translated access request to the object data storage service, via the provider network;
send to the client a response to the access request based at least in part on a response received from one of the non-relational database service or the object data storage service, wherein the response being sent to the client is formatted according to the network-based storage protocol.

7. The system of claim 6 , wherein object-backed block-based storage service is configured to implement an Internet Small Computer Interface (iSCSI) as the network-based storage protocol.

8. The system of claim 6 , wherein the access request is a write request directed to the particular data block,
wherein the translated access requests comprises an update request to update the cached data block entry for the particular data block maintained in the block cache non-relational database, and wherein to determine success or failure of the one or more translated access requests, the one or more storage protocol targets are configured to:
receive an acknowledgment of the update request from the non-relational database service indicating the success or failure of the update to complete.

9. The system of claim 6 ,
wherein the access request is a read request directed to the particular data block;
wherein in response to the receipt of the read request the one or more storage protocol targets are each further configured to determine a current storage location for the particular data block in the block cache of the non-relational database service or in the object data storage service;
wherein the translated access requests comprise a request to get the particular cached data block entry for the particular data block from the current storage location at the non-relational database service or a request to get the particular data block object for the particular data block from the object data storage service; and
wherein to send to the client the response to the access request formatted according to the network-based storage protocol, the one or more storage protocol targets are each configured to send the particular cached data block entry or the particular data block object.

10. The system of claim 9 , where to send to the client the response to the read request, the one or more storage protocol targets are each configured to send a first one received of either the particular cached data block entry from the non-relational database service or the particular data block object from the object data storage service.

11. The system of claim 6 , wherein the object-backed block-based storage service further comprises:
one or more cache management agents, configured to:
identify one or more of the plurality of cached data block entries to transfer to a respective one or more data block objects in the plurality of data block objects at

Assignees

• Amazon Tech Inc

Inventors

• Gorski Christopher Albert
• Branker Nicholas Scott

Classifications

• G06F16/289 Primary

  Object oriented databases · CPC title

• G06F12/0868 Primary

  Data transfer between cache memory and other subsystems, e.g. storage devices or host systems · CPC title

• G06F12/121

  using replacement algorithms · CPC title

• G06F12/128

  adapted to multidimensional cache systems, e.g. set-associative, multicache, multiset or multilevel · CPC title

• G06F16/172

  Caching, prefetching or hoarding of files · CPC title