Retention-based data management in a network-based data store

Patent metadata

Field	Value
Publication number	US-10412170-B2
Application number	US-201815912353-A
Country	US
Kind code	B2
Filing date	Mar 5, 2018
Priority date	Jun 29, 2015
Publication date	Sep 10, 2019
Grant date	Sep 10, 2019

Abstract

A network-based data store may implement retention-based management techniques for data stored at the network-based data store. When data is received for storage at the network-based data store, a retention time for the data may be determined. Storage locations at persistent storage devices of the network-based data store may be selected according to the retention time. The data may then be placed at the storage locations. When a request to delete data is received, retention times of co-located data may be evaluated to determine whether the deletion may be delayed. Delayed deletions may allow the data to be subsequently deleted with at least some of the co-located data. Repair operations to maintain the data according to a durability policy may be modified according to retention time for a data suffering a loss of redundancy.

First claim

What is claimed is:

1. A system, comprising:
one or more persistent storage devices for a network-based data store; and
a plurality of compute nodes that implement a control plane for the network-based data store, wherein the control plane is configured to:
receive a request to delete data at one or more storage locations of the network-based data store, wherein respective storage locations of a plurality of data stored at the network-based data store have been selected according to respective data retention times of individual ones of the plurality of data;
identify one or more respective retention times for other data stored at the one or more storage locations storing the data; and
delay deletion of the data such that the data is deleted with at least some of the other data based, at least in part, on the one or more respective retention times for the other data.

2. The system of claim 1 , wherein the one or more persistent storage devices are shingled magnetic recording devices.

3. The system of claim 2 , wherein the selection of one or more of the respective storage locations of the plurality of data stored at the network-based data store comprises an identified portion of a region of the one or more shingled magnetic recording devices that overlaps other data with longer respective retention times.

4. The system of claim 1 , wherein to delay deletion of the data, the control plane is configured to create a deletion marker, wherein the deletion marker indicates that the data is no longer available for access by clients of the network-based data store.

5. The system of claim 1 , wherein the control plane is further configured to:
receive another request to delete the other data at the one or more storage locations of the network-based data store;
delete the data and the other data.

6. The system of claim 1 , wherein the network-based data store is a storage service implemented as part of a provider network, and wherein the data is stored for a client of the storage service and associated with a particular customer account of the provider network.

7. A method, comprising:
performing, by one or more computing devices:
receiving a request to delete data stored in a network-based data store;
identifying respective retention times for other data stored at one or more storage locations at one or more persistent storage devices implemented as part of the network-based data store that also store the data; and
instructing deletion of the data based, at least in part, on an evaluation of the one or more respective retention times for the other data.

8. The method of claim 7 , wherein the one or more persistent storage devices are shingled magnetic recording devices.

9. The method of claim 8 , wherein the one or more storage locations are a portion of a region of the one or more shingled magnetic recording devices that overlaps different data with longer respective retention times.

10. The method of claim 7 , wherein instructing deletion of the data comprises creating a deletion marker indicating that the data is no longer available for access by clients of the network-based data store.

11. The method of claim 8 , wherein instructing deletion of the data further comprises scheduling the data for deletion as part of a batch deletion operation.

12. The method of claim 7 , comprising:
further performing, by one or more computing devices:
receiving another request to delete the other data stored at the one or more storage locations of the network-based data store;
deleting the data and the other data.

13. The method of claim 7 , wherein the network-based data store is a storage service implemented as part of a provider network, and wherein the data is stored for a client of the storage service and associated with a particular customer account of the provider network.

14. A non-transitory, computer-readable storage medium, storing program instructions that when executed by one or more computing devices cause the one or more computing devices to implement:
receiving a request to delete data at one or more storage locations of a network-based data store comprising one or more persistent storage devices, wherein respective storage locations of a plurality of data stored on the one or more persistent storage devices have been selected according to respective data retention times of individual ones of the plurality of data;
identifying one or more respective retention times for other data stored at the one or more storage locations storing the data; and
delaying deletion of the data such that the data is deleted with at least some of the other data based, at least in part, on the one or more respective retention times for the other data.

15. The non-transitory, computer-readable storage medium of claim 14 , wherein the one or more persistent storage devices are shingled magnetic recording devices.

16. The non-transitory, computer-readable storage medium of claim 15 , wherein the selection of one or more of the respective storage locations of the plurality of data stored at the network-based data store comprises an identified portion of a region of the one or more shingled magnetic recording devices that overlaps other data with longer respective retention times.

17. The non-transitory, computer-readable storage medium of claim 14 , wherein delaying deletion of the data comprises creating a deletion marker, wherein the deletion marker indicates that the data is no longer available for access by clients of the network-based data store.

18. The non-transitory, computer-readable storage medium of claim 14 , wherein the program instructions cause the one or more computing devices to further implement:
receiving another request to delete the other data at the one or more storage locations of the network-based data store;
deleting the data and the other data.

19. The non-transitory, computer-readable storage medium of claim 14 , wherein the network-based data store is a storage service implemented as part of a provider network, and wherein the data is stored for a client of the storage service and associated with a particular customer account of the provider network.

20. The non-transitory, computer-readable storage medium of claim 14 , wherein the network-based data store is an object storage service, and wherein the data is stored for a client of the object storage service and associated with a particular customer account.

Assignees

• Amazon Tech Inc

Inventors

• Davis Timothy James
• Patel Rajesh Shanker
• Marshall Bradley Eugene
• Collins Jonathan Robert

Classifications

• H04L67/1097 Primary

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F16/2365

  Ensuring data consistency and integrity · CPC title