Data storage system with enforced fencing

Patent metadata

Field	Value
Publication number	US-10484015-B2
Application number	US-201615392806-A
Country	US
Kind code	B2
Filing date	Dec 28, 2016
Priority date	Dec 28, 2016
Publication date	Nov 19, 2019
Grant date	Nov 19, 2019

Abstract

A data storage system includes multiple head nodes and data storage sleds. The data storage sleds include multiple mass storage devices and a sled controller. Respective ones of the head nodes are configured to obtain credentials for accessing particular portions of the mass storage devices of the data storage sleds. A sled controller of a data storage sled determines whether a head node attempting to perform a write on a mass storage device of a data storage sled that includes the sled controller is presenting with the write request a valid credential for accessing the mass storage devices of the data storage sled. If the credentials are valid, the sled controller causes the write to be performed and if the credentials are invalid, the sled controller returns a message to the head node indicating that it has been fenced off from the mass storage device.

First claim

What is claimed is:

1. A method comprising:
requesting, by a head node of a data storage system, credentials for accessing portions of mass storage devices of a plurality of data storage sleds of the data storage system, wherein the head node includes a sequence number for the head node with requests to the plurality of data storage sleds;
receiving, by the head node, credentials for accessing the portions, wherein the credentials are issued based, at least in part, on sled controllers of the data storage sleds determining the sequence number for the head node is greater than one or more respective greatest sequence numbers previously stored for the portions;
receiving, by the head node, a write request from a client of the data storage system;
writing, by the head node, data included with the write request to a data storage of the head node; and
requesting, by the head node, to write the data included with the write request to a plurality of mass storage devices in a plurality of data storage sleds of the data storage system, wherein requesting to write the data comprises:
presenting at least some of the respective credentials obtained from the plurality of data storage sleds; and
determining the respective credentials are valid credentials for accessing respective portions of the plurality of mass storage devices; and

writing the data included with the write request to the plurality of mass storage devices in the plurality of data storage sleds of the data storage system.

2. The method of claim 1 ,
wherein the sequence number is assigned to the head node from a data storage system control plane.

3. The method of claim 1 , further comprising:
subsequent to writing, by the head node, the data included with the write request to the data storage of the head node, causing the data included with the write request to be replicated to another head node of the data storage system.

4. The method of claim 3 , wherein said requesting to write the data included with the write request to a plurality of mass storage devices and said causing the data included with the write request to be replicated to another head node are performed asynchronously.

5. The method of claim 3 , wherein the head node initially performs a role of primary head node and the other head node initially performs a role of secondary head node, the method further comprising:
assuming, by the head node, the role of secondary head node based, at least in part, on an indication that one of the presented respective credentials is an invalid credential.

6. The method of claim 5 , further comprising:
assuming, by the other head node, the role of primary head node; and
requesting a new sequence number for the other head node acting as primary head node from a data storage system control plane.

7. The method of claim 6 , wherein the data storage system control plane is implemented on one or more head nodes of the data storage system, wherein the one or more head nodes and the plurality of data storage sleds are mounted in a common rack.

8. The method of claim 6 , wherein the data storage system control plane is implemented on one or more computing devices external to a rack that includes one or more head nodes of the data storage system and the plurality of data storage sleds of the data storage system.

9. The method of claim 5 , further comprising:
requesting, by the other head node acting as primary head node, access to the plurality of mass storage devices in the plurality of data storage sleds of the data storage system, wherein requesting access comprises presenting a sequence number assigned to the other head node acting as primary head node;
determining the sequence number is greater than a sequence number stored for the plurality of mass storage devices; and
issuing new respective credentials to the other head node acting as primary head node.

10. A data storage system comprising:
a plurality of head nodes; and
a plurality of data storage sleds comprising mass storage devices and sled controllers;
wherein the sled controllers of the respective data storage sleds are configured to:
issue a credential for accessing a portion of the mass storage devices of the respective data storage sled to a head node of the plurality of head nodes, if a sequence number included with the request is greater than a greatest sequence number previously stored for the portion of the mass storage devices and cause the sequence number included with the request to be stored for the portion of the mass storage devices; and
not issue the credential to the head node for accessing the portion of the mass storage devices, if the sequence number included with the request is not greater than the greatest sequence number previously stored for the portion of the mass storage devices;

wherein based, at least in part, on a request from a particular head node to write data on a particular portion of a particular mass storage device in a particular data storage sled, the sled controller of the particular data storage sled is configured to:
determine whether a credential included with the write request is a valid credential for accessing the particular portion of the particular mass storage device; and
based, at least in part, on determining the credential is a valid credential for the particular portion of the particular mass storage device, cause the requested write to be performed on the particular portion of the particular mass storage device; and
based, at least in part, on determining the credential is an invalid credential for the particular portion of the particular mass storage device, decline to perform the requested write.

11. The data storage system of claim 10 , wherein, for a part of a data volume, a first one of the plurality of head nodes functions as a primary head node and a second one of the plurality of head nodes functions as a secondary head node,
wherein the first head node is configured to store data included in a write request to a data store of the first head node, cause the data included in the write request to be replicated to the second head node, and cause the data included in the write request to be stored in portions of multiple mass storage devices,
wherein the first head node, upon receiving an indication that a sequence number associated with at least one of the portions of the multiple mass storage devices is greater than a sequence number issued to the first head node, assumes a role of secondary head node for the given part of the data volume.

12. A method comprising:
issuing to one or more head nodes of a data storage system one or more credentials for accessing one or more portions of mass storage devices of the data storage system, wherein said issuing comprises:
receiving a sequence number assigned to a particular head node;
determining the sequence number is greater than a greatest sequence number previously stored for a particular portion of a mass storage device for which the particular head node is seeking access; and
allocating a credential to the particular head node for accessing the particular portion of the mass storage device;

determining whether a given credential included with a write request from a particular head node of the data storage system is a valid credential for accessing a particular portion of a particular mass storage device of the data storage system;
determining the given credential is a valid credential for the particular portion of the particular mass storage device;
causing the requested write to be performed on the particular portion of the particular mass storage device;
determining whether another credential obtained from the data storage

Assignees

• Amazon Tech Inc

Inventors

• Kusters Norbert P
• Arumugam Nachiappan
• Watson Christopher Nathan
• Brooker Marc John
• Richardson David R
• Wei Danny
• Guthrie Ii John Luther
• Shalev Leah

Classifications

• G06F3/065

  Replication mechanisms · CPC title

• G06F11/2094

  Redundant storage or storage space (G06F11/2056 takes precedence) · CPC title

• G06F11/1076

  Parity data used in redundant arrays of independent storages, e.g. in RAID systems · CPC title

• H03M13/154 Primary

  Error and erasure correction, e.g. by using the error and erasure locator or Forney polynomial · CPC title

• G06F3/067 Primary

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title