Data system with flush views

Patent metadata

Field	Value
Publication number	US-12443349-B2
Application number	US-202117319020-A
Country	US
Kind code	B2
Filing date	May 12, 2021
Priority date	Feb 15, 2017
Publication date	Oct 14, 2025
Grant date	Oct 14, 2025

Abstract

A data storage system includes a head node and mass storage devices. The head node is configured to flush data stored in a storage of the head node to a set of the mass storage devices of the data storage system. A head node may flush both current version data and point-in-time version data to the set of mass storage devices. Also, the data storage system maintains an index that indicates storage locations of data for particular portions of a volume before and after the data is flushed to the set of mass storage devices. In some embodiments an index includes a current version reference for a volume or portion of a volume and one or more point-in-time snapshot references.

First claim

What is claimed is:

1. A system, comprising:
one or more computing devices configured to implement a block-based storage service configured to:
provide block-based storage volumes configured to be attached to respective compute instances provided by a virtual compute service of a provider network;
receive, from a client of the provider network, via an application programmatic interface (API), a request to implement snapshotting of a given one of the block-based storage volumes, wherein the request specifies:
a frequency-based criteria for generating one or more snapshots of a first portion of the given one of the block-based storage volumes, wherein each of the one or more snapshots represents a state of the first portion of the given one of the block-based storage volumes at a different fixed point-in-time; and
a write-based criteria for generating one or more other snapshots of a second portion of the given one of the block-based storage volumes, wherein each of the other one or more snapshots represents a state of the second portion of the given one of the block-based storage volumes at a different fixed point-in-time; and

implement snapshotting for the first portion of the given one of the block-based storage volumes and the second portion of the given one of the block-based storage volumes in accordance with the frequency-based criteria and the write-based criteria indicated by the client such that:
one or more snapshots are generated for the first portion of the given one of the block-based storage volumes based on the frequency-based criteria; and
one or more other snapshots are generated for the second portion of the given one of the block-based storage volumes based on the write-based criteria.

2. The system of claim 1 , wherein the block-based service is further configured to:
receive another request to implement snapshotting at a different given one of the block-based storage volumes, wherein the another request specifies:
a first criteria for generating one or more snapshots at a first portion of the different given one of the block-based storage volumes; and
a second criteria for generating one or more other snapshots at a second portion of the different given one of the block-based storage volumes;

wherein the first criteria is a first time-based frequency and the second criteria is a different time-based frequency,
wherein snapshotting is implemented such that snapshots are generated for the first portion of the different given one of the block-based storage volumes and the second portion of the different given one of the block-based storage volumes at different frequencies.

3. The system of claim 1 , wherein the request further specifies an offset range of the given one of the block-based storage volumes for which the write-based criteria for generating snapshots is to be applied.

4. The system of claim 3 , wherein snapshotting is implemented such that snapshots are generated for the second portion of the given one of the block-based storage volumes when a write is performed affecting the specified offset range corresponding to the first portion of the given one of the block-based storage volumes.

5. The system of claim 4 , wherein the frequency-based criteria is a time-based frequency,
wherein snapshotting is further implemented such that snapshots are generated for the first portion of the given one of the block-based storage volumes according to the time-based frequency.

6. The system of claim 4 , wherein the request further specifies a time range for which a write-based frequency is to be used to implement snapshotting for the second portion of the given one of the block-based storage volumes,
wherein snapshotting is implemented such that snapshots are generated for the second portion of the given one of the block-based storage volumes when a write is performed affecting the specified offset range corresponding to the second portion of the given one of the block-based storage volumes during the specified time range, and wherein snapshots are generated for the second portion of the given one of the block-based storage volumes according to a time-based frequency outside of the specified time range.

7. The system of claim 1 , wherein the one or more computing devices that implement the block-based storage comprise:
a log storage and an associated index indicating storage locations for volume data stored in the log storage, and

wherein to generate a snapshot of the given one of the block-based storage volumes, the one or more computing devices that implement the block-based storage are configured to:
update or add a reference for the given volume to the index such that the index includes a current version reference for the given one of the block-based storage volumes and a snapshot reference for the given one of the block-based storage volumes.

8. The system of claim 7 , wherein the one or more computing devices that implement the block-based storage are further configured to:
receive additional writes for the given volume;
store additional data included in the additional writes for at least one of the first portion of the given one of the block-based storage volumes or the second portion of the given one of the block-based storage volumes subsequent to generating one or more snapshots for the first portion of the given one of the block-based storage volumes or the second portion of the given one of the block-based storage volumes;
receive a request for a snapshot copy of the given one of the block-based storage volumes at a point-in-time of one of the one or more generated snapshots; and
generate a snapshot copy of the first or second portion of the given one of the block-based storage volumes at the requested point in time according to data stored at locations indicated by a given snapshot reference for the requested point in time snapshot.

9. The system of claim 8 , wherein the one or more computing devices that implement the block-based storage are further configured to:
cause at least some of the data and at least some of the additional data stored in the log storage to be written from the log storage to a set of mass storage devices; and
update the index for the log storage such that the current version reference and the snapshot version reference indicate locations in the set of mass storage devices at which the at least some data or additional data for the given one of the block-based storage volumes is stored.

10. The system of claim 8 , wherein the one or more computing devices that implement the block-based storage are further configured to:
export snapshot copies of the first portion of the given one of the block-based storage volumes or the second portion of the given one of the block-based storage volumes to an object-based storage system of the provider network.

11. A method, comprising:
providing a block-based storage volume;
receiving, via an application programmatic interface (API), a request to implement snapshotting for the block-based storage volume, wherein the request specifies:
a frequency-based snapshotting criteria for generating snapshots of a first portion of the block-based storage volume, wherein each of the snapshots represents a state of the first portion of the block-based storage volume at a different fixed point-in-time; and
a write-based snapshotting criteria for generating other snapshots of a second portion of the block-based storage volume, wherein each of the other snapshots represents a state of the second portion of the block-based storage volume at a different fixed point-in-time; and

implementing snapshotting of the block-based storage volume in accordance with the frequency-based snapshotting

Assignees

• Amazon Tech Inc

Inventors

• Kusters Norbert Paul
• Arumugam Nachiappan
• Podnozov Andre
• Agrawal Shobha
• Ramalingam Shreyas
• Wei Danny
• Richardson David R
• Brooker Marc John
• Watson Christopher Nathan
• Guthrie Ii John Luther
• Nankani Ravi

Classifications

• G06F11/2058

  using more than 2 mirrored copies · CPC title

• G06F11/2064

  while ensuring consistency · CPC title

• G06F2201/84

  Using snapshots, i.e. a logical point-in-time copy of the data · CPC title

• G06F11/2071

  using a plurality of controllers · CPC title

• G06F3/0683

  Plurality of storage devices · CPC title