Storage systems with adaptive erasure code generation

Patent metadata

Field	Value
Publication number	US-9292384-B2
Application number	US-201314125297-A
Country	US
Kind code	B2
Filing date	Aug 5, 2013
Priority date	Aug 5, 2013
Publication date	Mar 22, 2016
Grant date	Mar 22, 2016

Abstract

Apparatuses, methods and storage medium associated with generating erasure codes for data to be stored in a storage system. In embodiments, a method may include launching, by storage system, a plurality of instances of an erasure code generation module, based at least in part on hardware configuration of the storage system. Additionally, the method may further include setting, by the storage system, operational parameters of the plurality of instances of the erasure code generation module, based at least in part on current system load of the storage system. Further, the method may include operating, by the storage system, the plurality of instances of the erasure code generation module to generate erasure codes for data to be stored in the storage system, in accordance with the operational parameters set. Other embodiments may be described and claimed.

First claim

What is claimed is:

1. An apparatus for storing data, comprising:
a plurality of computer-storage nodes, wherein each computer-storage node includes at least one processor and at least one storage unit;
an erasure code generation module to generate erasure codes for data to be stored across a plurality of the storage units of the computer-storage nodes; and
a dispatch engine to:
launch one or more execution threads of the erasure code generation module to be operated in parallel by one or more processors of the computer-storage nodes to generate the erasure codes for data to be stored across a plurality of the storage units of the computer-storage nodes, wherein the number of execution threads of the erasure code generation module launched is based at least in part on hardware configuration of the computer-storage nodes; and
set one or more erasure code parameters for the plurality of execution threads of the erasure code generation module, wherein the one or more erasure code parameters are set based at least in part on current system workload of the computer-storage nodes.

2. The apparatus of claim 1 , wherein at least one of the processors comprises a plurality of processor cores.

3. The apparatus of claim 1 wherein the dispatch engine is to further retrieve the hardware configuration of the computer-storage nodes.

4. The apparatus of claim 1 , wherein the dispatch engine is to further receive, information about the current system workload.

5. The apparatus of claim 4 , further comprising a system workload monitor to monitor system workload, and provide information about current system workload to the dispatch engine.

6. The apparatus of claim 1 , wherein the one or more execution threads of the erasure code module are to cooperate and generate in parallel at least a first and a second parity of an erasure code from at least a first, a second and a third data block, and respectively store the first parity and the second parity at a first and a second storage units of a first and a second of the computer-storage nodes, with the first data block, the third data block, and the second data block respectively stored in a third, a fourth, and a fifth storage unit of a third, a fourth and a fifth of the computer-storage nodes.

7. One or more non-transitory computer-readable storage medium having a plurality of instructions, in response to execution of the instructions by a storage system, to cause a storage system to:
launch a plurality of execution threads of an erasure code generation module to be executed in parallel to generate erasure codes for data to be stored across a plurality of the storage units of the computer-storage nodes, wherein the launching is based at least in part on hardware configuration of the storage system;
set one or more erasure code parameters for the plurality of execution threads of the erasure code generation module, wherein the setting is based at least in part on current system workload of the storage system; and
operate, the plurality of execution threads of the erasure code generation module to generate the erasure codes for data to be stored across the storage system, in accordance with the erasure code parameters set.

8. The storage medium of claim 7 , wherein the storage system is a scale out storage system having a plurality of computer-storage nodes, wherein at least one of the computer-storage nodes includes a microprocessor having a plurality of processor cores.

9. The storage medium of claim 7 , wherein launch comprises launch, using a dispatch engine of the storage system, the plurality of execution threads of the erasure code generation module, based at least in part on hardware configuration of the storage system, and wherein the storage system to further retrieve, using the dispatch engine, the hardware configuration of the storage system.

10. The storage medium of claim 7 , wherein set erasure code parameters comprises set one or more buffer sizes, erasure code sizes, number of k symbols to be encoded into n symbols.

11. The storage medium of claim 7 , wherein set comprises set, using a dispatch engine of the storage system, the erasure code parameters of the plurality of execution threads of the erasure code generation module, based at least in part on current system workload of the storage system; and wherein the storage system to further comprises receive, using the dispatch engine, information about the current system workload.

12. The storage medium of claim 11 , wherein receive comprises receive the information about the current system workload, using the dispatch engine, from a system workload monitor of the storage system, wherein the storage system to further monitor current system workload, using the system workload monitor.

13. The storage medium of claim 7 , wherein the storage system comprises a scale out storage system having a plurality of computer-storage nodes; wherein operate comprises operate the plurality of execution threads of the erasure code generation module to generate at least a first and a second parity of an erasure code from at least a first, a second and a third data block, and respectively store the first parity, the second parity, the first data block, the third data block, and the second data block in a first, a second, a third, a fourth, and a fifth of the computer-storage nodes.

Assignees

• Intel Corp

Inventors

• Jin Jun
• Jin Zhonghui
• Qiao Nan Stan
• Di Zeyu

Classifications

• H03M13/353

  Adaptation to the channel · CPC title

• H03M13/373

  with erasure correction and erasure determination, e.g. for packet loss recovery or setting of erasures for the decoding of Reed-Solomon codes · CPC title

• H03M13/05 Primary

  using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits {(H03M13/2906 takes precedence)} · CPC title

• G06F11/1076 Primary

  Parity data used in redundant arrays of independent storages, e.g. in RAID systems · CPC title

• H03M13/2906 Primary

  using block codes (H03M13/2957 takes precedence) · CPC title