Data storage system with multi-tier control plane

Patent metadata

Field	Value
Publication number	US-10509601-B2
Application number	US-201615392878-A
Country	US
Kind code	B2
Filing date	Dec 28, 2016
Priority date	Dec 28, 2016
Publication date	Dec 17, 2019
Grant date	Dec 17, 2019

Abstract

A data storage system includes multiple data storage units and a zonal control plane. The zonal control plane assigns volumes to respective ones of the data storage units. The data storage units include multiple head nodes and data storage sleds. At least one of the head nodes implements a local control plane for the data storage unit. Also, the head nodes of each data storage unit are configured to service read and write requests directed to one or more volumes serviced by the data storage unit independent of the zonal control plane.

First claim

What is claimed is:

1. A data storage system comprising:
a plurality of data storage units, wherein respective ones of the data storage units comprise:
a plurality of head nodes; and
a plurality of data storage sleds; and

one or more computing devices external to the plurality of data storage units configured to implement a zonal control plane for partially controlling storage operations related to the plurality of data storage units,
wherein, based, at least in part, on a volume creation request, the zonal control plane assigns a particular one of the data storage units to service a volume requested by the volume creation request; and
wherein for the particular one of the plurality of data storage units, two or more head nodes implement a local control plane comprising an allocation service and a distributed data store, distributed across two or more of the head nodes of the particular data storage unit;
wherein the local control plane is configured to assign one of the plurality of head nodes of the particular data storage unit to function as a primary head node for the volume, wherein the primary head is node configured to service read requests and write requests directed to the volume separate from the local control plane and the zonal control plane,
wherein the allocation service of the local control plane is configured to allocate portions of space to respective ones of the plurality of head nodes of the particular data storage unit from amongst the plurality of data storage sleds of the particular data storage unit,
wherein, in response to a failure one of the two or more head nodes implementing the local control plane, the local control plane is configured to fail over to a remaining one of the two or more head nodes and is configured to access the distributed data store, and
wherein in response to the failure of the one of the two or more head nodes implementing the local control plane, the primary head node is configured to continue to service read requests and write requests directed to the volume and write data for the volume to a portion of space allocated to the primary head node for the volume from amongst the plurality of data storage sleds of the particular data storage unit.

2. The data storage system of claim 1 , wherein for the volume assigned to the particular one of the data storage units by the zonal control plane, the local control plane of the particular data storage unit is configured to assign another head node of the particular data storage unit to function as a secondary head node for the volume.

3. The data storage system of claim 2 , wherein for the volume, the local control plane of the particular data storage unit selects the primary head node from amongst any of the plurality of head nodes of the particular data storage unit and the local control plane selects the secondary head node from amongst the remaining head nodes that are not functioning as the primary head node for the volume.

4. The data storage system of claim 3 , wherein a given head node of the plurality of head nodes of the particular data storage unit is configured to function as a primary head node for one or more volumes and is configured to function as a secondary head node for one or more other volumes.

5. The data storage system of claim 2 , wherein the primary head node for the volume is configured to receive, from the local control plane, allocation of respective portions of a plurality of respective mass storage devices of the plurality of data storage sleds for storing data of the given volume;
wherein the allocation service is configured to track allocation of portions of space on the mass storage devices in the plurality of data storage sleds of the particular data storage unit and allocate portions of space on the mass storage devices.

6. A data storage system comprising:
a data storage unit comprising:
a plurality of head nodes; and
a plurality of data storage sleds,
wherein two or more of the head nodes of the data storage unit implement a local control plane for the data storage unit configured to communicate with a zonal control plane, wherein the local control plane comprises an allocation service and a distributed data store distributed across two or more of the head nodes,

wherein, based, at least in part, on a volume creation request, the local control plane is configured to receive an assignment from the zonal control plane to service a volume requested by the volume creation request, wherein the zonal control plane at least partially controls storage operations related to the data storage unit and one or more additional data storage units,
wherein the local control plane is configured to assign one of the plurality of head nodes to function as a primary head node for the volume, the primary head node is configured to service read requests and write requests directed to the volume separate from the local control plane and the zonal control plane,
wherein the allocation service of the local control plane is configured to allocate portions of space to respective ones of the plurality of head nodes from amongst the plurality of data storage sleds of the data storage unit,
wherein, in response to a failure one of the two or more head nodes implementing the local control plane, the local control plane is configured to fail over to a remaining one of the two or more head nodes and is configured to access the distributed data store, and
wherein in response to the failure of the one of the two or more head nodes implementing the local control plane, the primary head node is configured to continue to service read requests and write requests directed to the volume and write data for the volume to a portion of space allocated to the primary head node for the volume from amongst the plurality of data storage sleds of the data storage unit.

7. The data storage system of claim 6 , wherein the allocation service is configured to track allocation of portions of space on mass storage devices of the data storage sleds of the data storage unit and allocate portions of space on the mass storage devices in the plurality of data storage sleds of the data storage unit.

8. The data storage system of claim 7 , wherein for a given volume, a head node functioning as primary head node for the volume stores data of the volume on portions of mass storage devices in different ones of the plurality of data storage sleds that are allocated to the primary head node.

9. The data storage system of claim 6 , wherein head nodes of the data storage unit are configured to fail over between one or more head nodes implementing the local control plane independent of the zonal control plane.

10. The data storage system of claim 6 , wherein for a given volume, based, at least in part on a failure of a head node functioning as primary head node for the given volume or a failure of a head node functioning as secondary head node for the given volume, the local control plane is configured to assign another head node of the data storage unit to function as a secondary head node for the given volume.

11. The data storage system of claim 6 , wherein the local control plane is configured to allocate a set of portions of space on mass storage devices in multiple data storage sleds of the data storage unit to a given volume,
wherein, a head node functioning as a primary head node for the given volume selects portions of space on the mass storage devices to store data of the given volume from the set of portions.

12. The data storage system of claim 11 , wherein, based, at least in part, on a determination that durability for the given volume would be less than a threshold durability if space on ma

Assignees

• Amazon Tech Inc

Inventors

• Kusters Norbert P
• Arumugam Nachiappan
• Watson Christopher Nathan
• Brooker Marc John
• Richardson David R
• Wei Danny
• Guthrie Ii John Luther

Classifications

• G06F3/065

  Replication mechanisms · CPC title

• G06F11/1076

  Parity data used in redundant arrays of independent storages, e.g. in RAID systems · CPC title

• G06F3/0619

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F3/0665 Primary

  at area level, e.g. provisioning of virtual or logical volumes · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title