Data storage system with multi-tier control plane

Patent metadata

Field	Value
Publication number	US-11237772-B2
Application number	US-201916714589-A
Country	US
Kind code	B2
Filing date	Dec 13, 2019
Priority date	Dec 28, 2016
Publication date	Feb 1, 2022
Grant date	Feb 1, 2022

Abstract

A data storage system includes multiple data storage units and a zonal control plane. The zonal control plane assigns volumes to respective ones of the data storage units. The data storage units include multiple head nodes and data storage sleds. At least one of the head nodes implements a local control plane for the data storage unit. Also, the head nodes of each data storage unit are configured to service read and write requests directed to one or more volumes serviced by the data storage unit independent of the zonal control plane.

First claim

What is claimed is:

1. A method, comprising:
servicing, by a designated-primary head node of a data storage unit that comprises a designated-secondary head node configured to replicate data of the designated-primary head node, and one or more data storage sleds for storing data flushed from head nodes of the data storage unit, read requests and write requests directed to a volume stored in the data storage unit; and
generating, by a local control plane of the data storage unit, a recommendation of a sandbox of specified storage, associated with the volume, in the data storage sleds;
wherein the recommendation indicates availability of the sandbox of specified storage in the data storage sleds for selection, by the designated-primary head node, for data storage during a data flush from the designated-primary head node to the data storage sleds.

2. The method of claim 1 , further comprising:
transmitting the recommendation to the designated-primary head node.

3. The method of claim 1 , further comprising:
selecting, by the designated-primary head node, from the sandbox of specified storage, storage to store data during the data flush from the designated-primary head node to the data storage sleds.

4. The method of claim 1 , further comprising:
flushing, by the designated-primary head node, the data to columns within the sandbox of specified storage without requesting, for the flush, extent allocation from the local control plane.

5. The method of claim 1 , further comprising:
determining:
that insufficient space is available in the sandbox of specified storage, or
that a particular placement of the data would cause a data durability of the saved data to fall below of minimum required durability for the volume; and

selecting, based on the determining, one or more columns outside the sandbox of specified storage to write data for the volume.

6. The method of claim 1 , further comprising:
determining that communication between the local control plane and the designated-primary head node has been lost; and
continuing, based on the determining, to flush data from the designated-primary head node to the data storage sleds.

7. The method of claim 1 , further comprising:
collecting performance metrics from one or more of the data storage sleds or the head nodes; and
issuing, based on the performance metrics, a sandbox update to the designated-primary head node;
wherein the recommendation is based at least in part on the sandbox update.

8. The method of claim 1 , further comprising:
identifying a heat collision comprising multiple of the head nodes attempting to access a same data storage sled at a same time; and
to avoid the identified heat collision:
issuing the sandbox update, or
generating and transmitting the sandbox recommendation.

9. A data storage system, comprising:
a data storage unit assigned to service read requests and write requests directed to a volume partition stored in the data storage unit, the data storage unit comprising:
a plurality of head nodes comprising:
a designated-primary head node configured to service the volume partition; and
one or more data storage sleds for storing data flushed from the head nodes;

wherein the designated-primary head node is configured to:
receive a recommendation that indicates availability of a sandbox of specified storage, allocated to the volume and in the data storage sleds, for selection, by the designated-primary head node, for data storage during a data flush from the designated-primary head node to the data storage sleds;
select, from the sandbox of specified storage in the data storage sleds that is allocated to the volume, one or more portions of storage to store data during the data flush from the designated-primary head node to the data storage sleds; and
flush data to the selected storage.

10. The data storage system of claim 9 , wherein the designated-primary head node is configured to:
determine:
that communication between a local control plane implemented by one of the head nodes and the designated-primary head node has been lost,
that a particular placement of the data would cause a data durability of the saved data to fall below of minimum required durability for the volume, or
that insufficient space is available in the specified storage of the sandbox; and

flush, based on the determination, data to columns outside of the specified storage of the sandbox.

11. The data storage system of claim 9 , wherein the designated-primary head node is further configured to flush the data to columns within the sandbox of specified storage without requesting, from a local control plane implemented by one of the head nodes, and for the flush, an allocation of memory in the storage sleds.

12. The data storage system of claim 11 , further comprising:
an extent allocation service configured to allocate extents as sets of columns of space on mass storage devices in different sleds that each store some of the flushed data;
wherein the sandbox comprises extents that include corresponding columns in multiple mass storage devices; and
wherein the recommendation is generated by a local control plane implemented by one of the head nodes and transmitted to the designated-primary head node and indicates availability of extents for selection by the designated-primary head node to select for data storage during the data flush from the designated-primary head node to the data storage sleds.

13. The data storage system of claim 11 , wherein the designated-primary head node is configured to:
determine that communication between the local control plane and the designated-primary head node has been lost; and
continue, based on the determination, to flush data from the designated-primary head node to the data storage sleds.

14. The data storage system of claim 11 , wherein:
the sandbox comprises extents that include corresponding columns in multiple mass storage devices.

15. The data storage system of claim 11 , wherein the local control plane is configured to:
collect performance metrics from one or more of the data storage sleds or the head nodes; and
issue, based on the performance metrics, a sandbox update to the designated-primary head node;
wherein the recommendation is based at least in part on the sandbox update.

16. A data storage system, comprising:
a head node of a data storage unit assigned to service read requests and write requests directed to a volume partition stored in a data storage unit, the data storage unit comprising:
respective designated-primary head nodes configured to service respective volume partitions stored in the data storage unit;
respective designated-secondary head nodes configured to replicate data of the corresponding designated-primary head nodes;
and data storage sleds for storing data flushed from the designated-primary head nodes,

wherein the head node is configured to implement a local control plane for the data storage unit, configured to:
generate and transmit a recommendation of a sandbox of specified storage, associated with a respective volume partition, in the data storage sleds, to a respective designated-primary head node for the respective volume partition;
wherein the recommendation indicates availability of the sandbox of specified storage in the data storage sleds for selection, by the respective designated-primary head node, for data storage during a data flush from the respective designated-primary head node to the data storage sleds.

17. The data storage system of claim 16 ,
wherein the respective designated-p

Assignees

• Amazon Tech Inc

Inventors

• Kusters Norbert P
• Arumugam Nachiappan
• Watson Christopher Nathan
• Brooker Marc John
• Richardson David R
• Wei Danny
• Guthrie Ii John Luther

Classifications

• G06F3/065

  Replication mechanisms · CPC title

• G06F11/1076

  Parity data used in redundant arrays of independent storages, e.g. in RAID systems · CPC title

• G06F3/0665 Primary

  at area level, e.g. provisioning of virtual or logical volumes · CPC title

• G06F3/0619

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title