Oversubscribed storage extents with on-demand page allocation

Patent metadata

Field	Value
Publication number	US-9779015-B1
Application number	US-201414231106-A
Country	US
Kind code	B1
Filing date	Mar 31, 2014
Priority date	Mar 31, 2014
Publication date	Oct 3, 2017
Grant date	Oct 3, 2017

Abstract

In response to receiving a write request directed to a particular logical block of a storage object, a page of free space (sufficient to accommodate the payload of the write request, but smaller in size than the logical block) of a particular extent that has been selected to store contents of the logical block is allocated. The current size of the extent is smaller than the combined sizes of logical blocks that are mapped to the extent. The page is modified in accordance with a payload indicated in the write request. In response to a subsequent write request directed to the particular extent, a determination is made that the particular extent would violate a free space threshold criterion if the payload of the write request were accommodated, and an extent expansion operation is initiated.

First claim

What is claimed is:

1. A system, comprising:
one or more computing devices configured to:
designate a particular extent of a distributed multi-tenant storage service to store contents of a particular logical block of a storage object, wherein the particular extent is configured to store contents of a plurality of logical blocks of one or more storage objects including the particular logical block in accordance with a physical storage oversubscription policy;
in response to a first write request directed to the particular logical block, wherein the first write request indicates a first payload that is smaller than a size of the particular logical block,
allocate one or more pages of free space of the extent to the particular logical block to accommodate the first payload, wherein a combined size of the one or more pages is smaller than the size of the particular logical block; and
modify the one or more pages in accordance with the first payload; and

in response to a subsequent write request directed to the particular extent,
determine that the particular extent comprises insufficient free storage space to accommodate a second payload indicated in the subsequent write request without violating a free space threshold criterion associated with the oversubscription policy;
initiate an extent expansion operation; and
prior to a completion of the extent expansion operation,
allocate at least one page of free space of the extent to accommodate the second payload; and
modify the at least one page in accordance with the second payload.

2. The system as recited in claim 1 , wherein the one or more computing devices are further configured to:
modify one or more parameters of the oversubscription policy, including the free space threshold criterion, based on one or more metrics of storage use.

3. The system as recited in claim 1 , wherein the storage object comprises one of: (a) a metadata structure, or (b) a file.

4. The system as recited in claim 1 , wherein the one or more computing devices are further configured to:
rearrange, in accordance with a performance optimization policy, contents of a set of pages of the particular extent at a particular physical storage device.

5. The system as recited in claim 1 , wherein the first and subsequent write requests are formatted according to one of: (a) NFS (Network File System), or (b) SMB (Server Message Block).

6. A method, comprising:
performing, by one or more computing devices:
selecting a particular extent of a storage service to store contents of a particular logical block of a storage object, wherein the particular extent is designated to store contents of a plurality of logical blocks of one or more storage objects, wherein a combined size of the plurality of logical blocks exceeds a current size of the particular extent;
in response to receiving a particular write request directed to the particular logical block, indicating a first payload,
allocating one or more pages of free space of the extent to the particular logical block to accommodate the first payload; and
modifying the one or more pages in accordance with the first payload; and

in response to receiving a subsequent write request directed to the particular logical block,
determining that the particular extent comprises insufficient free storage space to accommodate a second payload indicated in the subsequent write request without violating a free space threshold criterion; and
initiating an extent expansion operation; and

prior to a completion of the extent expansion operation, allocating a page of free space of the extent to the particular logical block to accommodate the second payload.

7. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:
modifying one or more parameters of the oversubscription policy, including the free space threshold criterion, based on one or more metrics of storage use.

8. The method as recited in claim 6 , wherein said modifying the one or more pages in accordance with the first payload comprises modifying contents of a first storage device, and wherein said extent expansion operation comprises utilizing a different physical storage device.

9. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:
modifying said free space threshold criterion based on one or more metrics of storage use.

10. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:
modifying said free space threshold criterion based on an analysis of names of the one or more storage objects.

11. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:
storing a plurality of replicas of the particular extent at respective nodes of a provider network in accordance with a data durability policy, including a master replica instantiated at a first data center and a non-master replica instantiated at a different data center;
initiating a modification of the one or more pages at the master replica; and
propagating the modification of the one or more pages to a remainder of the plurality of replicas.

12. The method as recited in claim 6 , wherein the storage object comprises a different logical block with a different block size than the particular logical block.

13. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:
rearranging, in accordance with a performance optimization policy, contents of a set of pages of the particular extent at a particular physical storage device.

14. The method as recited in claim 6 , wherein the storage object comprises a plurality of logical blocks, further comprising performing, by the one or more computing devices:
selecting a different extent of the storage service to store contents of the different logical block.

15. The method as recited in claim 6 , wherein the first and subsequent write requests are formatted according to an industry-standard file system interface.

16. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:
selecting a first extent, located at a first storage device at a first node of the storage service, as a candidate for transfer to a different storage location to enable expansion of a second extent at the first node;
identifying the different storage location for contents of the first extent, wherein the different storage location comprises at least one of: (a) a portion of a second storage device at the first node, or (b) a portion of a third storage device at a different node of the storage service;
copying contents of the first extent to the different storage location; and
releasing storage space allocated to the first extent at the first storage device.

17. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors:
in response to receiving a particular write request directed to a particular logical block of a storage object,
allocate one or more pages of free space of a particular extent selected to store contents of the particular logical block to accommodate a particular payload indicated in the particular write request, wherein a current size of the particular extent exceeds a combined size of a plurality of logical blocks whose contents are mapped to the particular extent; and
modify the one or more pages in accordance with the particular payload; and

in response to receiving a subsequent write request direc

Assignees

• Amazon Tech Inc

Inventors

• Oikarinen Matti Juhani
• Vincent Pradeep
• Frigo Matteo

Classifications

• G06F3/0644 Primary

  Management of space entities, e.g. partitions, extents, pools · CPC title

• G06F12/02 Primary

  Addressing or allocation; Relocation (program address sequencing G06F9/00; arrangements for selecting an address in a digital store G11C8/00) · CPC title

• G06F3/0608

  Saving storage space on storage systems · CPC title

• G06F3/0683

  Plurality of storage devices · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title