Data mobility, accessibility, and consistency in a data storage system

Patent metadata

Field	Value
Publication number	US-10860547-B2
Application number	US-201816152259-A
Country	US
Kind code	B2
Filing date	Oct 4, 2018
Priority date	Apr 23, 2014
Publication date	Dec 8, 2020
Grant date	Dec 8, 2020

Abstract

A transactional block storage system is provided which is capable of supporting a single-phase commit for data writes specifying a protected storage unit. The storage system includes a data storage map that logically links the protected data storage unit to two or more block storage units associated with a layer of the protected data storage unit. The storage system also includes an address abstraction layer which translates write requests to the block storage units and resolves whether those write requests are atomically committed to the storage system in a single phase transaction. The address abstraction layer is further configured to detected when a block storage unit becomes unavailable during a transaction and create a cleaning kit for that block in order to prevent data loss. Additionally, the address abstraction layer facilitates moving, copying, and merging of block storage units without global locking in the storage system.

First claim

We claim:

1. A data storage system for providing access to data in a file system over a network, comprising:
a plurality of data storage devices; and
one or more computers, including:
one or more memories for storing instructions; and
one or more processors that execute the instructions to perform actions, including:
accessing data stored in a plurality of storage units, wherein each storage unit includes a plurality of block storage units (bstore), and wherein the file system employs one or more received write requests to select one or more of the plurality of storage units as protected storage units (pstore);
providing a map for each pstore and two or more corresponding bstores, wherein the map associates an address with each pstore, and wherein the one or more received write requests and associated pstore address are employed to access one or more of the two of more corresponding bstores;
persistently storing metrics of aggregated values at each directory that corresponds to at least one file object
employing each received write request to generate one or more translated write requests, wherein each translated write request indicates a different one of the corresponding bstores; and
in response to unavailability of the different one of the two or more bstores, performing further actions, including:
generating a cleaning kit at an available bstore on a node that is separate from the one different unavailable bstore on another node, wherein the cleaning kit includes each in-flight transaction that is uncommitted to one or more bstores;
adding a new bstore to an available data storage device containing the one different unavailable bstore, wherein the new bstore stores data intended for the unavailable bstore that is provided by the cleaning kit; and
updating the unavailable bstore with data included in the new bstore when the unavailable bstore is available.

2. The data storage system of claim 1 , further comprising:
collectively representing each translated write requests, as being positive or negative based on one or more commit responses provided by the one or more of the corresponding bstores.

3. The data storage system of claim 1 , wherein providing the map further comprises including a copy of the map in an address abstraction layer that also includes an address for each pstore, and wherein the address abstraction layer communicates with a kernel layer that employs the received write request and associated pstore address to access one or more of the corresponding bstores.

4. The data storage system of claim 1 , further comprising:
employing the receiving of each write request by the two or more bstores to subsequently provide corresponding commit requests to the two or more bstores without receiving acknowledgment for each write request.

5. The data storage system of claim 1 , further comprising:
employing a null or unknown commit response from one or more bstores to provide the cleaning kit that includes data to bring the one or more bstores back to a known state, wherein the included data is less than a full backup copy of the bstore.

6. The data storage system of claim 1 , further comprising:
writing one or more commit requests to a write ahead log for each corresponding bstore.

7. The data storage system of claim 1 , further comprising:
in response to a startup state, designating one node to query each bstore for a list of each log entry for each corresponding write ahead log, wherein the list is employed to build a transaction status table.

8. A method for providing access to stored data over a network, comprising:
employing one or more processors to execute instructions to perform actions, including:
accessing data stored in a plurality of storage units, wherein each storage unit includes a plurality of block storage units (bstore), and wherein the file system employs one or more received write requests to select one or more of the plurality of storage units as protected storage units (pstore);
providing a map for each pstore and two or more corresponding bstores, wherein the map associates an address with each pstore, and wherein the one or more received write requests and associated pstore address are employed to access one or more of the two of more corresponding bstores;
employing each received write request to generate one or more translated write requests, wherein each translated write request indicates a different one of the corresponding bstores; and
in response to unavailability of the different one of the two or more bstores, performing further actions, including:
generating a cleaning kit at an available bstore on a node that is separate from the one different unavailable bstore on another node, wherein the cleaning kit includes each in-flight transaction that is uncommitted to one or more bstores;
adding a new bstore to an available data storage device containing the one different unavailable bstore, wherein the new bstore stores data intended for the unavailable bstore that is provided by the cleaning kit; and
updating the unavailable bstore with data included in the new bstore when the unavailable bstore is available.

9. The method of claim 8 , further comprising:
collectively representing each translated write requests, as being positive or negative based on one or more commit responses provided by the one or more of the corresponding bstores.

10. The method of claim 8 , wherein providing the map further comprises including a copy of the map in an address abstraction layer that also includes an address for each pstore, and wherein the address abstraction layer communicates with a kernel layer that employs the received write request and associated pstore address to access one or more of the corresponding bstores.

11. The method of claim 8 , further comprising:
employing the receiving of each write request by the two or more bstores to subsequently provide corresponding commit requests to the two or more bstores without receiving acknowledgment for each write request.

12. The method of claim 8 , further comprising:
employing a null or unknown commit response from one or more bstores to provide the cleaning kit that includes data to bring the one or more bstores back to a known state, wherein the included data is less than a full backup copy of the bstore.

13. The method of claim 8 , further comprising:
writing one or more commit requests to a write ahead log for each corresponding bstore.

14. The method of claim 8 , further comprising:
in response to a startup state, designating one node to query each bstore for a list of each log entry for each corresponding write ahead log, wherein the list is employed to build a transaction status table.

15. A non-transitory data storage media that includes instructions for providing access to stored data over a network, wherein execution of the instructions by one or more processors performs actions, including:
accessing data stored in a plurality of storage units, wherein each storage unit includes a plurality of block storage units (bstore), and wherein the file system employs one or more received write requests to select one or more of the plurality of storage units as protected storage units (pstore);
providing a map for each pstore and two or more corresponding bstores, wherein the map associates an address with each pstore, and wherein the one or more received write requests and associated pstore address are employed to access one or more of the two of more corresponding bstores;
employing each received write request to generate one or more translated write requests, wherein each translated write re

Assignees

• Qumulo Inc

Inventors

• Passey Aaron
• Fachan Neal
• Godman Peter

Classifications

• G06F16/2358

  Change logging, detection, and notification (replication G06F16/27) · CPC title

• G06F9/467

  Transactional memory (G06F9/528 takes precedence) · CPC title

• G06F16/185 Primary

  Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof (details of archiving G06F16/11) · CPC title