Fast geo recovery method for elastic cloud storage

Patent metadata

Field	Value
Publication number	US-10579490-B2
Application number	US-201715636470-A
Country	US
Kind code	B2
Filing date	Jun 28, 2017
Priority date	Dec 23, 2016
Publication date	Mar 3, 2020
Grant date	Mar 3, 2020

Abstract

One embodiment is related to a method for remote replication recovery, comprising: determining that a damaged data chunk at a first zone of a cloud storage system is not recoverable locally; determining one or more data fragments of the damaged data chunk that are to be recovered with remote replication recovery based on data stored at a second zone and at a third zone of the cloud storage system, wherein the damaged data chunk comprises a plurality of data fragments; recovering the one or more data fragments of the damaged data chunk with remote replication recovery based on the data stored at the second zone and at the third zone of the cloud storage system; and repairing the damaged data chunk with the recovered data fragments at the first zone.

First claim

What is claimed is:

1. A method for remote replication recovery, comprising:
determining that a damaged data chunk at a first zone of a cloud storage system is not recoverable locally, wherein each data chunk corresponds to storage space of a predetermined fixed size and comprises a plurality of data fragments that are protected with one or more coding fragments generated based on erasure coding;
determining one or more data fragments of the damaged data chunk that are to be recovered with remote replication recovery based on data stored at a second zone and at a third zone of the cloud storage system;
recovering the one or more data fragments of the damaged data chunk with remote replication recovery based on the data stored at the second zone and at the third zone of the cloud storage system, wherein during the remote replication recovery the data fragments to be recovered are each identified within the respective data chunk with a fragment offset and a fragment size; and
subsequent to recovering the one or more data fragments of the damaged data chunk, repairing the damaged data chunk with the recovered data fragments at the first zone.

2. The method of claim 1 , wherein determining that the damaged data chunk at the first zone of the cloud storage system is not recoverable locally further comprises determining that the damaged data chunk is not recoverable using erasure coding based on data stored within the first zone.

3. The method of claim 1 , wherein the first, second, and third zones are geographically distributed.

4. The method of claim 1 , wherein the one or more data fragments that are to be recovered with remote replication recovery comprise all of invalid data fragments of the damaged data chunk.

5. The method of claim 1 , wherein the one or more data fragments that are to be recovered with remote replication recovery are determined so that remaining invalid data fragments not recovered with remote replication recovery are recoverable locally at the first zone with erasure coding.

6. The method of claim 5 , further comprising recovering the remaining invalid data fragments of the damaged data chunk locally at the first zone with erasure coding.

7. The method of claim 1 , wherein recovering the one or more data fragments of the damaged data chunk with remote replication recovery further comprises performing a bitwise exclusive or (XOR) operation on at least part of the data stored at the second zone and at the third zone of the cloud storage system.

8. The method of claim 1 , further comprising protecting the repaired data chunk with erasure coding at the first zone.

9. A non-transitory machine-readable medium having instructions stored therein which, when executed by a processor, cause the processor to perform testing operations, the operations comprising:
determining that a damaged data chunk at a first zone of a cloud storage system is not recoverable locally, wherein each data chunk corresponds to storage space of a predetermined fixed size and comprises a plurality of data fragments that are protected with one or more coding fragments generated based on erasure coding;
determining one or more data fragments of the damaged data chunk that are to be recovered with remote replication recovery based on data stored at a second zone and at a third zone of the cloud storage system;
recovering the one or more data fragments of the damaged data chunk with remote replication recovery based on the data stored at the second zone and at the third zone of the cloud storage system, wherein during the remote replication recovery the data fragments to be recovered are each identified within the respective data chunk with a fragment offset and a fragment size; and
subsequent to recovering the one or more data fragments of the damaged data chunk, repairing the damaged data chunk with the recovered data fragments at the first zone.

10. The non-transitory machine-readable medium of claim 9 , wherein determining that the damaged data chunk at the first zone of the cloud storage system is not recoverable locally further comprises determining that the damaged data chunk is not recoverable using erasure coding based on data stored within the first zone.

11. The non-transitory machine-readable medium of claim 9 , wherein the first, second, and third zones are geographically distributed.

12. The non-transitory machine-readable medium of claim 9 , wherein the one or more data fragments that are to be recovered with remote replication recovery comprise all of invalid data fragments of the damaged data chunk.

13. The non-transitory machine-readable medium of claim 9 , wherein the one or more data fragments that are to be recovered with remote replication recovery are determined so that remaining invalid data fragments not recovered with remote replication recovery are recoverable locally at the first zone with erasure coding.

14. The non-transitory machine-readable medium of claim 13 , wherein the operations further comprise recovering the remaining invalid data fragments of the damaged data chunk locally at the first zone with erasure coding.

15. The non-transitory machine-readable medium of claim 9 , wherein recovering the one or more data fragments of the damaged data chunk with remote replication recovery further comprises performing a bitwise exclusive or (XOR) operation on at least part of the data stored at the second zone and at the third zone of the cloud storage system.

16. The non-transitory machine-readable medium of claim 9 , wherein the operations further comprise protecting the repaired data chunk with erasure coding at the first zone.

17. A data processing system, comprising:
a processor; and
a memory coupled to the processor storing instructions which, when executed by the processor, cause the processor to perform testing operations, the operations including
determining that a damaged data chunk at a first zone of a cloud storage system is not recoverable locally, wherein each data chunk corresponds to storage space of a predetermined fixed size and comprises a plurality of data fragments that are protected with one or more coding fragments generated based on erasure coding;
determining one or more data fragments of the damaged data chunk that are to be recovered with remote replication recovery based on data stored at a second zone and at a third zone of the cloud storage system;
recovering the one or more data fragments of the damaged data chunk with remote replication recovery based on the data stored at the second zone and at the third zone of the cloud storage system, wherein during the remote replication recovery the data fragments to be recovered are each identified within the respective data chunk with a fragment offset and a fragment size; and
subsequent to recovering the one or more data fragments of the damaged data chunk, repairing the damaged data chunk with the recovered data fragments at the first zone.

18. The data processing system of claim 17 , wherein determining that the damaged data chunk at the first zone of the cloud storage system is not recoverable locally further comprises determining that the damaged data chunk is not recoverable using erasure coding based on data stored within the first zone.

19. The data processing system of claim 17 , wherein the first, second, and third zones are geographically distributed.

20. The data processing system of claim 17 , wherein the one or more data fragments that are to be recovered with remote replication recovery comprise all of invalid data frag

Assignees

• Emc Ip Holding Co Llc

Inventors

• Danilov Mikhail
• Buinov Konstantin
• Karpenkov Sergey
• Trusov Maxim S
• Zakharov Kirill

Classifications

• G06F2201/82

  Solving problems relating to consistency · CPC title

• H03M13/3761

  using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes · CPC title

• G06F2201/805

  Real-time · CPC title

• H04L67/1095

  Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes · CPC title

• G06F11/2082

  Data synchronisation · CPC title