High performance storage system

Patent metadata

Field	Value
Publication number	US-2017242587-A1
Application number	US-201615048043-A
Country	US
Kind code	A1
Filing date	Feb 19, 2016
Priority date	Feb 19, 2016
Publication date	Aug 24, 2017
Grant date	—

Abstract

A data storage structure, comprising: a plurality of storage units, each comprising: a storage media; and a library executive configured to manage the storage media. The structure further comprises a buffer connected to a controller, the controller comprising: a host interface configured to receive the instruction from the host machine; an object aggregator configured to combine the plurality of data objects into a data segment; a persistent write buffer configured to store the data segment; a persistent map configured to identify a location of each of the plurality of objects in the data segment; an erasure coder configured to encode the data segment into an erasure code; a destager configured to transfer the data segment from the persistent write buffer to the storage media in a given storage unit; and a library controller configured to communicate with the library executive in the given storage unit.

First claim

What is claimed is:

1 . A data storage structure, comprising:
a plurality of storage units, each comprising:
a storage media comprising a plurality of data blocks; and
a library executive configured to manage the storage media;
a controller configured to communicate with each of the storage units over a network and comprising an object aggregator configured to combine a plurality of data objects into a data segment; and an erasure code configured to protect against concurrent loss of: at least one storage unit, at least one storage media residing in an alternate storage unit, and at least one data block residing in an alternate storage media.

2 . The data storage structure of claim 1 , the controller comprising:
a host interface configured to receive instructions from a host machine; a buffer configured to store the data segment; and a map configured to identify a location of each of the plurality of data objects in the data segment.

3 . The data storage structure of claim 2 , the controller further comprising:
an erasure coder configured to encode the data segment into the erasure code; a destager configured to transfer the data segment from the buffer to the storage media in a given storage unit; and a library controller configured to communicate with the library executive in the given storage unit.

4 . The data storage structure of claim 3 , wherein at least one parity element is written into the erasure code such that it depends only on at least one prior written element in the erasure code.

5 . The data storage structure of claim 3 , wherein an integrity check is computed and stored, by the erasure coder, within each element in the erasure code.

6 . The data storage structure of claim 3 , the object aggregator further configured to encode data objects in the data segment so the map can be reconstructed by reading segments stored in the storage media.

7 . The data storage structure of claim 2 , wherein a size of the buffer is configured to be dynamically allocated to meet dynamic workloads.

8 . The data storage structure of claim 3 , wherein the destager is further configured to make the data segment available to the erasure coder in a sequential fashion so only a single pass through the data segment is needed.

9 . The data storage structure of claim 1 , wherein:
the erasure code is further configured to be stored across a plurality of media storages in the plurality of storage units; and the erasure code is further configured so a given storage unit may reconstruct a single data block residing in the given storage unit without communicating with the controller.

10 . A method, comprising:
storing a plurality of data objects in a buffer; combining the plurality of data objects into a data segment; storing in a map a location of each data object in the data segment; and encoding the plurality of data segments into an erasure code; wherein the erasure code is encoded protect against concurrent loss of: at least one storage unit, at least one storage media residing in an alternate storage unit, and at least one data block residing in an alternate storage media.

11 . The method of claim 10 , wherein encoding further comprises writing at least one parity element in the erasure code wherein the parity element depends only on at least one prior written element in the erasure code.

12 . The method of claim 10 , wherein encoding further comprises computing and storing an integrity check value in each element in the erasure code.

13 . The method of claim 10 , wherein data objects in the data segment are encoded in a self-describing format within the data segment.

14 . The method of claim 10 , further comprising:
transferring the erasure code to more than one storage media residing among a plurality of storage units.

15 . The method of claim 10 , wherein a size of the buffer is dynamically allocated to meet dynamic workloads.

16 . The method of claim 10 , wherein encoding further comprises:
providing the data segment in a sequential order using a plurality of data streams to ensure only one pass through the data segment is required.

17 . A system comprising a computer processor, a computer-readable hardware storage device, and program code embodied with the computer-readable hardware storage device for execution by the computer processor to implement a method comprising:
storing a plurality of data objects in a buffer; combining the plurality of data objects into a data segment; storing in a map a location of each data object in the data segment; providing the data segment to an erasure coder in a sequential order using a plurality of data streams to ensure only one pass through the data segment is required; and encoding the plurality of data segments into an erasure code; wherein the erasure code is encoded protect against concurrent loss of: at least one storage unit, at least one storage media residing in an alternate storage unit, and at least one data block residing in an alternate storage media.

18 . The system of claim 17 , wherein encoding further comprises writing at least one parity element in the erasure code wherein the parity element depends only on at least one prior written element in the erasure code.

19 . The system of claim 18 , wherein encoding further comprises computing and storing an integrity check value in each element in the erasure code.

20 . The system of claim 17 , wherein the buffer is a persistent write buffer.

Assignees

• IBM

Inventors

• Blaum Mario
• Hetzler Steven R
• Hineman Wayne C
• Rees Robert M

Classifications

• G06F3/0652

  Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket · CPC title

• G06F3/0656

  Data buffering arrangements · CPC title

• G06F12/0891

  using clearing, invalidating or resetting means · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/062

  Securing storage systems · CPC title