Efficient recovery of storage gateway cached volumes

Patent metadata

Field	Value
Publication number	US-9268651-B1
Application number	US-201213665685-A
Country	US
Kind code	B1
Filing date	Oct 31, 2012
Priority date	Oct 31, 2012
Publication date	Feb 23, 2016
Grant date	Feb 23, 2016

Abstract

Methods and apparatus for efficient recovery of cached volumes at storage gateways are disclosed. To recover, after an unplanned shutdown, a storage gateway appliance configured to cache chunks of a storage object, chunk metadata corresponding to a particular chunk is read into an in-memory metadata region from a first metadata location. Based on analysis of the chunk metadata, a validation requirement indication for the particular chunk is stored, and the chunk is designated as being accessible for client I/O requests. In response to receiving a subsequent I/O request targeted to the particular chunk, the chunk metadata is validated using a different metadata location prior to performing the requested I/O operation.

First claim

The invention claimed is:

1. A method, comprising:
in response to a determination to bring a storage appliance into a running state after an unplanned shutdown, wherein the appliance is configured to cache a plurality of data chunks of a storage object of a remote storage service,
reading into an in-memory metadata region, from a first metadata location of the appliance, chunk metadata corresponding to a particular data chunk of the plurality of data chunks;
based at least in part on an analysis of a portion of the chunk metadata, storing an indication of a validation requirement in the in-memory metadata region for the particular data chunk;
designating the particular data chunk as being accessible for input/output (I/O) requests after the indication of the validation requirement has been stored; and

in response to receiving, after said designating, an I/O request targeted to at least a portion of the particular data chunk, validating at least a portion of the chunk metadata using a different metadata storage location of the appliance, prior to performing one or more I/O operations indicated in the I/O request.

2. The method as recited in claim 1 , wherein the first metadata location comprises a portion of a first space of a storage device, wherein the first space is designated to store contiguous metadata for one or more cached data chunks of the storage object, including the particular data chunk; and wherein the different metadata location comprises an inline metadata section stored in a second space of the storage device, wherein the second space is configured to store the at least a portion of the particular data chunk.

3. The method as recited in claim 1 , wherein the analysis comprises determining whether a caching state of the particular data chunk, as indicated within the chunk metadata, belongs to a particular set of caching states.

4. The method as recited in claim 1 , further comprising:
in response to the determination to bring the storage appliance into the running state,
reading into the in-memory metadata region, from the first metadata location, second chunk metadata corresponding to a second data chunk of the plurality of data chunks;
based at least in part on an analysis of the second chunk metadata, updating at least a portion of the second chunk metadata in the in-memory metadata region using contents of an inline metadata location of the appliance prior to accepting I/O requests targeted at the second data chunk.

5. The method as recited in claim 1 , wherein the appliance comprises an upload buffer configured to store modified data of the storage object to be uploaded asynchronously to the remote storage service, further comprising:
in response to the determination to bring the storage appliance into the running state, adding an entry to the upload buffer, wherein the entry comprises contents one or more data blocks of the particular data chunk.

6. A system, comprising:
one or more hardware computing devices configured to:
in response to a determination to bring a storage appliance into a running state after an unplanned shutdown, wherein the appliance is configured to cache a plurality of data chunks of a storage object of a remote storage service,
read into an in-memory metadata region, from a first metadata location of the appliance, chunk metadata corresponding to a particular data chunk of the plurality of data chunks;
based at least in part on an analysis of a portion of the chunk metadata, store an indication of a validation requirement for the particular data chunk;
designate the particular data chunk as being accessible for input/output (I/O) requests after the indication of the validation requirement has been stored; and

in response to receiving, after the designation of the particular data chunk, an I/O request targeted to at least a portion of the particular data chunk, validate at least a portion of the chunk metadata using a different metadata storage location of the appliance, prior to a completion of one or more I/O operations indicated in the I/O request.

7. The system as recited in claim 6 , wherein the indication of the validation requirement is stored in the in-memory metadata region.

8. The system as recited in claim 6 , wherein the first metadata location comprises a portion of a first space on a storage device, wherein the first space is designated to store contiguous metadata for one or more cached data chunks of the storage object, including the particular data chunk; and wherein the different metadata location comprises an inline metadata section stored in a second space of the storage device, wherein the second space is configured to store the at least a portion of the particular data chunk.

9. The system as recited in claim 8 , wherein the particular data chunk comprises a plurality of data blocks, wherein each data block of the plurality of data blocks has a corresponding inline metadata section, and wherein, within the second space, the plurality of data blocks of the particular data chunk are arranged in an alternating sequence with the corresponding inline metadata sections.

10. The system as recited in claim 6 , wherein the analysis comprises a determination of whether a caching state of the particular data chunk, as indicated within the chunk metadata, belongs to a particular set of caching states.

11. The system as recited in claim 6 , wherein the one or more hardware computing devices are further configured to:
in response to the determination to bring the storage appliance into the running state,
read into the in-memory metadata region, from the first metadata location of the appliance, second chunk metadata corresponding to a second data chunk of the plurality of data chunks;
update, in response to a detection of a particular indicator within the second chunk metadata, at least a portion of the second chunk metadata in the in-memory metadata region using contents of an inline metadata location of the appliance prior to accepting I/O requests targeted at the second data chunk.

12. The system as recited in claim 11 , wherein the particular indicator comprises a metadata entry stored in the first metadata location by a cache evictor configured to free at least some cache storage of the appliance.

13. The system as recited in claim 11 , wherein to update the at least a portion of the second chunk metadata, the one or more hardware computing devices are further configured to:
compare an instance identifier stored in the second chunk metadata, indicative of a validity of a data block of the second data chunk, with an instance identifier stored in the inline metadata location.

14. The system as recited in claim 6 , wherein the appliance comprises an upload buffer configured to store modified data of the storage object to be uploaded asynchronously to the remote storage service, wherein the one or more hardware computing devices are further configured to:
in response to the determination to bring the storage appliance into the running state, add an entry to the upload buffer, wherein the entry comprises contents one or more data blocks of the particular data chunk.

15. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors:
in response to a determination to bring a storage appliance into a running state after an unplanned shutdown, wherein the appliance is configured to cache a plurality of data chunks of a storage object of a remote storage service,
read into an in-memory metadata region, from a first metadata location of the appliance, chunk metadata corresponding to a pa

Assignees

• Amazon Tech Inc

Inventors

• Salyers David Carl
• Khetrapal Ankur
• Vincent Pradeep
• Patiejunas Kestutis

Classifications

• G06F2212/154

  Networked environment · CPC title

• G06F12/0868

  Data transfer between cache memory and other subsystems, e.g. storage devices or host systems · CPC title

• G06F2212/284

  being distributed · CPC title

• G06F11/1435

  using file system or storage system metadata · CPC title

• G06F12/0871

  Allocation or management of cache space · CPC title