Cached volumes at storage gateways

Patent metadata

Field	Value
Publication number	US-9268652-B1
Application number	US-201213665708-A
Country	US
Kind code	B1
Filing date	Oct 31, 2012
Priority date	Oct 31, 2012
Publication date	Feb 23, 2016
Grant date	Feb 23, 2016

Abstract

Methods and apparatus for supporting cached volumes at storage gateways are disclosed. A storage gateway appliance is configured to cache at least a portion of a storage object of a remote storage service at local storage devices. In response to a client's write request, directed to at least a portion of a data chunk of the storage object, the appliance stores a data modification indicated in the write request at a storage device, and asynchronously uploads the modification to the storage service. In response to a client's read request, directed to a different portion of the data chunk, the appliance downloads the requested data from the storage service to the storage device, and provides the requested data to the client.

First claim

What is claimed is:

1. A method, comprising:
designating a first space on a storage device of a storage appliance to (a) cache at least a portion of one or more data chunks of a storage object of a remote storage service, wherein each chunk of the one or more data chunks comprises one or more data blocks and (b) store inline metadata for the data blocks of the one or more data chunks, wherein the inline metadata for a particular data block comprises a plurality of metadata elements;
designating a second space on the storage device to store contiguous metadata for the one more data chunks of the storage object, wherein at least a portion of the contiguous metadata is based at least in part on a portion of inline metadata stored in the first space;
updating a particular portion of the contiguous metadata based at least in part on a modification of at least one element of inline metadata, wherein said updating the particular portion of the contiguous metadata is performed asynchronously with respect to the modification of the at least one element of inline metadata; and
after a controlled shutdown and restart of the storage appliance, determining a state of a particular data chunk of the one or more data chunks using at least the particular portion of the contiguous metadata, and enabling client access to the particular data chunk dependent upon a result of said determining.

2. The method as recited in claim 1 , further comprising:
determining, using an in-memory version of metadata associated with the particular data chunk, whether a particular I/O request directed to at least a portion of the particular data chunk results in a cache miss, wherein at least a portion of the in-memory version corresponds to at least a portion of inline metadata stored for data blocks of the particular data chunk.

3. The method as recited in claim 1 , wherein the first space comprises an alternating sequence of cached data blocks of the particular data chunk and inline metadata sections corresponding to the cached data blocks, wherein each inline metadata section comprises one or more metadata elements, wherein the modification of the at least one element of inline metadata is performed in response to a particular I/O request directed at a particular data block of the particular data chunk, further comprising:
identifying an offset within the first space at which a particular element of inline metadata corresponding to the particular data block is stored, based at least in part on an offset of the particular data block within the particular data chunk.

4. The method as recited in claim 1 , further comprising:
receiving an I/O request comprising a modification directed to a particular data block of the particular data chunk;
modifying the particular data block within the first space in accordance with the I/O request;
adding an entry to an upload buffer configured for asynchronous uploads of modified data to the remote storage service, wherein the entry indicates modifications performed on the particular data block in response to the I/O request; and
verifying that modifications indicated in the entry have been uploaded to the remote storage service prior to replacing the particular data block in the first storage space.

5. The method as recited in claim 1 , further comprising:
receiving a read request directed to a particular data block of the particular data chunk;
downloading at least a portion of the particular data chunk, comprising the particular data block, from the remote storage service;
determining whether the at least a portion of the particular data chunk is to be merged with one or more cached data blocks present in the first space;
in response to determining that a merge is to be performed, merging the at least a portion of the particular data chunk with the one or more cached data blocks prior to storing the at least a portion of the particular data chunk in the first space; and
extracting the particular data block from the at least a portion of the particular data chunk and providing the particular data block in response to the read request.

6. A system, comprising
one or more computing devices; and
one or more memories storing program instructions that, when executed by at least one of the one or more computing devices, configure the at least one of the one or more computing devices to:
designate a first storage space of a storage appliance to (a) cache at least a portion of one or more data chunks of a storage object of a remote storage service and (b) store inline metadata for the one or more data chunks;
designate a second storage space of the storage appliance to store contiguous metadata for the one more data chunks of the storage object;
update a portion of the contiguous metadata based at least in part on a modification of at least a portion of inline metadata, wherein said update of the portion of the contiguous metadata is performed asynchronously with respect to the modification of the at least a portion of inline metadata; and
after a restart of the storage appliance, determine a state of the at least a portion of a particular data chunk using the portion of the contiguous metadata, and enable client access to the at least a portion of the particular data chunk.

7. The system as recited in claim 6 , wherein the one or more computing devices are further configured to:
determine, using an in-memory version of metadata associated with the particular data chunk, whether an I/O request directed to at least a portion of the particular data chunk results in a cache miss, wherein at least a portion of the in-memory version corresponds to at least a portion of inline metadata stored for the particular data chunk.

8. The system as recited in claim 6 , wherein the particular data chunk comprises a plurality of data blocks, wherein the first storage space comprises an alternating sequence of cached data blocks of the particular data chunk and inline metadata sections corresponding to the cached data blocks, wherein to modify the at least a portion of inline metadata in response to an I/O request directed to at least a particular data block of the plurality of data blocks, the one or more computing devices are configured to:
identify an offset within the first storage space at which the at least a portion of inline metadata is stored, based at least in part on an offset of the particular data block within the particular data chunk.

9. The system as recited in claim 8 , wherein to modify the at least a portion of inline metadata, the one or more computing devices are configured to:
identify a first inline metadata section adjacent to the particular data block, and a different inline metadata section adjacent to the particular data block; and
modify the first inline metadata section and the second inline metadata section prior to providing a response to the I/O request.

10. The system as recited in claim 6 , wherein the particular data chunk comprises a plurality of data blocks, wherein the one or more computing devices are further configured to:
receive a write request directed to at least a particular data block of the plurality of data blocks,
modify the particular data block within the first storage space in accordance with the write request;
add an entry to an upload buffer configured for asynchronous uploads of modified data to the remote storage service, wherein the entry indicates modifications performed on the particular data block; and
verify that modifications indicated in the entry have been uploaded to the remote storage service prior to replacing the particular data block in the first storage space.

11. The system as recited in claim 10 , wherein the storage

Assignees

• Amazon Tech Inc

Inventors

• Salyers David Carl
• Vincent Pradeep
• Khetrapal Ankur
• Patiejunas Kestutis

Classifications

• G06F12/0871

  Allocation or management of cache space · CPC title

• G06F2212/154

  Networked environment · CPC title

• G06F2212/60

  Details of cache memory · CPC title

• G06F2212/468

  The specific object being partially cached · CPC title

• G06F12/0804

  with main memory updating (G06F12/0806 takes precedence) · CPC title