Data mobility, accessibility, and consistency in a data storage system

Patent metadata

Field	Value
Publication number	US-10095708-B2
Application number	US-201514658015-A
Country	US
Kind code	B2
Filing date	Mar 13, 2015
Priority date	Apr 23, 2014
Publication date	Oct 9, 2018
Grant date	Oct 9, 2018

Abstract

A transactional block storage system is provided which is capable of supporting a single-phase commit for data writes specifying a protected storage unit. The storage system includes a data storage map that logically links the protected data storage unit to two or more block storage units associated with a layer of the protected data storage unit. The storage system also includes an address abstraction layer which translates write requests to the block storage units and resolves whether those write requests are atomically committed to the storage system in a single phase transaction. The address abstraction layer is further configured to detected when a block storage unit becomes unavailable during a transaction and create a cleaning kit for that block in order to prevent data loss. Additionally, the address abstraction layer facilitates moving, copying, and merging of block storage units without global locking in the storage system.

First claim

We claim:

1. A data storage system for providing access to data over a network, comprising:
a plurality of data storage devices; and
one or more computers, including:
one or more memories for storing instructions; and
one or more processors that execute the instructions to perform actions, including:
instantiating a client interface, by an application layer, to access data stored in a plurality of storage units, wherein each storage unit comprises a plurality of layers that include a plurality of block storage units (bstore) on the plurality of hardware data storage devices, wherein the file system employs a received write request to specify one or more of the storage units as protected (pstore), and;
instantiating a map that corresponds to each pstore with two or more block storage units (bstore), wherein a copy of the map is included in an address abstraction layer that also includes an address for each pstore, and wherein the address abstraction layer communicates with a kernel layer that employs the received write request and associated pstore address to access one or more of the corresponding bstores; and
employing the address abstraction layer to perform further actions, comprising:
employing each received write request to generate one or more translated write requests, wherein each translated write request specifies a different one of the two or more block storage units that corresponds to each protected data storage volume;
resolving a transaction, collectively representing the one or more translated write requests, as being positive or negative based on commit responses from each write request to the two or more block storage units, wherein the one or more write requests are atomically committed; and
when one of the two or more block storage units becomes unavailable during the single-phase transaction, performing further actions, including:
adding a new block storage unit to one layer of an available data storage device containing the unavailable block storage unit, wherein the new block storage unit stores data intended for the unavailable block storage unit; and
updating the unavailable block storage unit with data included in the new block storage unit when the unavailable block storage becomes available.

2. The data storage system of claim 1 , wherein the transaction is resolved as single-phase commit actions to improve performance of the data storage system.

3. The data storage system of claim 1 , wherein a commit response includes one of a positive response, a negative response, or a null response.

4. The data storage system of claim 1 , wherein the transaction is resolved to be a positive transaction when one or more of the write requests returns a positive commit response and none of the other one or more write requests returns a negative commit response.

5. The data storage system of claim 1 , wherein each write request originates from a client and provides a logical address for storing data, and wherein the two or more bstores are indicated by the logical address.

6. The data storage system of claim 1 , further comprising:
when the transaction is resolved as negative, terminating the transaction, and providing a notification that the one or more write requests are incomplete; and
when the transaction is resolved as positive, providing a notification that the one or more write requests are complete.

7. The data storage system of claim 1 , further comprising:
logging each commit response and each resolved transaction, wherein a positive response is logged as a successful commit, a negative response is logged as an unsuccessful commit, and a null is logged as a commit for which no response is received.

8. The data storage system of claim 1 , further comprising removing the new block storage unit from the data storage system when the unavailable block storage unit is updated.

9. A computer for providing access to data stored on a plurality of data storage devices over a network, comprising:
one or more memories for storing instructions; and
one or more processors that execute the instructions to perform actions, including:
instantiating a client interface, by an application layer, to access data stored in a plurality of storage units, wherein each storage unit comprises a plurality of layers that include a plurality of block storage units (bstore) on the plurality of hardware data storage devices, wherein the file system employs a received write request to specify one or more of the storage units as protected (pstore), and;
instantiating a map that corresponds to each pstore with two or more block storage units (bstore), wherein a copy of the map is included in an address abstraction layer that also includes an address for each pstore, and wherein the address abstraction layer communicates with a kernel layer that employs the received write request and associated pstore address to access one or more of the corresponding bstores; and
employing the address abstraction layer to perform further actions, comprising:
employing each received write request to generate one or more translated write requests, wherein each translated write request specifies a different one of the two or more block storage units that corresponds to each protected data storage volume;
resolving a transaction, collectively representing the one or more translated write requests, as being positive or negative based on commit responses from each write request to the two or more block storage units, wherein the one or more write requests are atomically committed; and
when one of the two or more block storage units becomes unavailable during the single-phase transaction, performing further actions, including:
adding a new block storage unit to one layer of an available data storage device containing the unavailable block storage unit, wherein the new block storage unit stores data intended for the unavailable block storage unit; and
updating the unavailable block storage unit with data included in the new block storage unit when the unavailable block storage becomes available.

10. The computer of claim 9 , wherein the transaction is resolved as single-phase commit actions to improve performance in accessing data.

11. The computer of claim 9 , wherein a commit response includes one of a positive response, a negative response, or a null response.

12. The computer of claim 9 , wherein the transaction is resolved to be a positive transaction when one or more of the write requests returns a positive commit response and none of the other one or more write requests returns a negative commit response.

13. The computer of claim 9 , wherein each write request originates from a client and provides a logical address for storing data, and wherein the two or more bstores are indicated by the logical address.

14. The computer of claim 9 , further comprising:
when the transaction is resolved as negative, terminating the transaction, and providing a notification that the one or more write requests are incomplete; and
when the transaction is resolved as positive, providing a notification that the one or more write requests are complete.

15. The computer of claim 9 , further comprising:
logging each commit response and each resolved transaction, wherein a positive response is logged as a successful commit, a negative response is logged as an unsuccessful commit, and a null is logged as a commit for which no response is received.

16. The computer of claim 9 , further comprising removing the new block storage unit from the data storage system when the unavailable block storage unit is updated.

Assignees

• Qumulo Inc

Inventors

• Passey Aaron
• Fachan Neal
• Godman Peter

Classifications

• G06F9/467

  Transactional memory (G06F9/528 takes precedence) · CPC title

• G06F16/2358

  Change logging, detection, and notification (replication G06F16/27) · CPC title

• G06F16/185 Primary

  Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof (details of archiving G06F16/11) · CPC title

• G06F17/30221 Primary

  Physics · mapped topic

• G06F17/30368

  Physics · mapped topic