Resiliency fragment tiering

Patent metadata

Field	Value
Publication number	US-9817715-B2
Application number	US-201514696302-A
Country	US
Kind code	B2
Filing date	Apr 24, 2015
Priority date	Apr 24, 2015
Publication date	Nov 14, 2017
Grant date	Nov 14, 2017

Abstract

Technology is disclosed for a data storage architecture for providing enhanced storage resiliency for a data object. The data storage architecture can be implemented in a single-tier configuration and/or a multi-tier configuration. In the single-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data fragments, which are stored across many storage devices. In the multi-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data segments, which are sent to one or more tiers of storage nodes and at least one latent storage. Each of the storage nodes further encodes the data segment to generate many data fragments representing the data segment, which are stored across many storage devices associated with the storage node. The I/O operations for rebuilding the data in case of device failures is spread across many storage devices, which minimizes the wear of a given storage device.

First claim

I claim:

1. A method comprising:
a front-end subsystem erasure coding an object into n fragments based on a storage resiliency value defined by a service level objective;
determining that a first plurality of the n fragments will be stored in a first storage tier, wherein a number of the first plurality of the n fragments is equal to or greater than k, wherein k is a minimum number of fragments sufficient to reconstruct the object;
determining that a second plurality of the n fragments will be stored in a second storage tier, wherein the second storage tier comprises latent storage, wherein the second plurality of then fragments consists of protection fragments; and
updating storage layout information to indicate locations of the fragments in the first and the second storage tiers.

2. The method of claim 1 , further comprising:
in response to an attempt to reconstruct the object, determining that available ones of the first plurality of the n fragments are insufficient to reconstruct the object; and
retrieving a sufficient number of the second plurality of the n fragments from the second storage tier to reconstruct the object with available ones of the first plurality of the n fragments from the first storage tier, wherein the sufficient number is according to the erasure coding.

3. The method of claim 2 , further comprising:
generating additional protection fragments to replenish the sufficient number of the second plurality of fragments used to reconstruct the object; and
transmitting the additional protection fragments for storing in the second storage tier.

4. The method of claim 1 , wherein the second storage tier is at least one of lower cost storage than the first storage tier and accessed via a lower throughput data access than the first storage tier.

5. The method of claim 1 , wherein a number of the second plurality of fragments is based on an estimated number of failures in the first storage tier between hardware refreshes of the first storage tier.

6. The method of claim 1 further comprising proactively replacing lost fragments of the first plurality of the n fragments using the second plurality of the n fragments although remaining ones of the first plurality of n fragments are sufficient to reconstruct the object.

7. The method of claim 1 , wherein n is greater than several hundred and the first plurality of n fragments includes redundant fragments of the object.

8. A storage system comprising:
a processor;
a plurality of storage interfaces; and
a computer-readable medium comprising instructions executable by the processor to cause the storage system to,
erasure code an object into n fragments that satisfies a storage resiliency value defined by a service level objective;
determine that a first plurality of the n fragments will be stored in a first storage tier, wherein a number of the first plurality of the n fragments is equal to or greater than k, wherein k is a minimum number of fragments sufficient to reconstruct the object;
determine that a second plurality of the n fragments will be stored in a second storage tier, wherein the second storage tier is latent storage, wherein the second plurality of the fragments consists of protection fragments; and
update storage layout information to indicate location of the fragments in the first and the second storage tiers.

9. The storage system of claim 8 , wherein the computer-readable medium further comprises instructions to:
determine that available ones of the first plurality of the n fragments are not sufficient to reconstruct the object; and
retrieve a sufficient number of the second plurality of the n fragments from the second storage tier to reconstruct the object with available ones of the first plurality of the n fragments from the first storage tier, wherein the sufficient number is according to the erasure coding.

10. The storage system of claim 9 , wherein the computer-readable medium further comprises instructions to:
generate additional protection fragments to replenish the sufficient number of the second plurality of fragments used to reconstruct the object; and
transmit, via a first set of the plurality of storage interfaces of a secondary data access path, the additional protection fragments for storing in the second storage tier, wherein a second set of the plurality of storage interfaces corresponds to a primary data access path of the first storage tier.

11. The storage system of claim 8 , wherein a primary data access path to the first storage tier comprises a first set of the plurality of storage interfaces and a secondary data access path to the second storage tier comprises a second set of the plurality of storage interfaces.

12. The storage system of claim 8 , wherein a number of the second plurality of the n fragments is based on an estimated number of failures in the first storage tier between hardware refreshes of the first storage tier.

13. The storage system of claim 8 , wherein the instructions further comprise instructions executable by the processor to cause the storage system to proactively replace lost fragments of the first plurality of the n fragments using the second plurality of the n fragments although remaining ones of the first plurality of n fragments are sufficient to reconstruct the object.

14. The storage system of claim 8 , wherein n is greater than several hundred and the first plurality of n fragments includes redundant fragments of the object.

15. The storage system of claim 8 , wherein the instructions to determine that the second plurality of the n fragments will be stored in the second storage tier comprises the instructions executable by the processor to cause the storage system to select the second storage tier for the second plurality of the n fragments based on the second storage tier comprising latent storage.

16. A non-transitory computer-readable medium comprising instructions for storing protection fragments of erasure coded objects in latent storage, the instructions to:
erasure code an object into n fragments based on a storage resiliency value defined by a service level objective;
determine that a first plurality of the n fragments will be stored in a first storage tier, wherein a number of the first plurality of the n fragments is equal to or greater than k, wherein k is a minimum number of fragments sufficient to reconstruct the object;
determine that a second plurality of the n fragments will be stored in a second storage tier, wherein the second storage tier comprises the latent storage, wherein the second plurality of the n fragments consists of protection fragments; and
update storage layout information to indicate location of the fragments in the first and the second storage tiers.

17. The computer-readable medium of claim 16 further comprising instructions:
determine that available ones of the first plurality of the n fragments are not sufficient to reconstruct the object; and
retrieve a sufficient number of the second plurality of the n fragments from the second storage tier to reconstruct the object with available ones of the first plurality of the n fragments from the first storage tier, wherein the sufficient number is according to the erasure coding.

18. The computer-readable medium of claim 16 further comprising instructions to proactively replace lost fragments of the first plurality of the n fragments using the second plurality of the n fragments although remaining ones of the first plurality of n fragments are sufficient to reconstruct the object.

19. The computer-rea

Assignees

• Netapp Inc

Inventors

• Slik David

Classifications

• G06F11/1076 Primary

  Parity data used in redundant arrays of independent storages, e.g. in RAID systems · CPC title

• G06F3/0689

  Disk arrays, e.g. RAID, JBOD · CPC title

• H03M13/373

  with erasure correction and erasure determination, e.g. for packet loss recovery or setting of erasures for the decoding of Reed-Solomon codes · CPC title

• G06F3/0619

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F3/064

  Management of blocks · CPC title