Hybrid intra-cluster migration of data between storage devices using chunk usage efficiency

Patent metadata

Field	Value
Publication number	US-11023129-B2
Application number	US-201816207754-A
Country	US
Kind code	B2
Filing date	Dec 3, 2018
Priority date	Dec 3, 2018
Publication date	Jun 1, 2021
Grant date	Jun 1, 2021

Abstract

Hybrid intra-cluster migration of data in an elastic cloud storage (ECS) environment is disclosed herein. A system comprises a processor and a memory that stores executable instructions that, when executed by the processor, perform operations that include moving a first data chunk from a first storage device to a second storage device during a data migration, based on a first determination that a first use efficiency of the first data chunk satisfies a defined use efficiency threshold. The operations also include moving a first data segment from a second data chunk in the first storage device to a third data chunk in the second storage device during the data migration, based on a second determination that a second use efficiency of the second data chunk fails to satisfy the defined use efficiency threshold. The first data segment includes data that is open for new writes and a second data segment includes data that is not open for new writes. After data is moved, capacity of the first data chuck is recovered. Moving the first data chunk preserves a chunk identifier of the first data chunk.

First claim

What is claimed is:

1. A method, comprising:
determining, by a processor of a computer system, a first use efficiency of a first data chunk of a first storage device satisfies a threshold usage level based on a first determination that the first data chunk is to be migrated to a second storage device;
determining, by the processor, that a second use efficiency of a second data chunk of the first storage device and a third use efficiency of a third data chunk of the first storage device fail to satisfy the threshold usage level based on a second determination that the second data chunk and the third data chunk are to be migrated to the second storage device;
moving, by the processor, the first data chunk from the first storage device to the second storage device; and
moving, by the processor, first live data of the second data chunk and second live data of the third data chunk into a fourth data chunk of the second storage device, wherein the moving the first data chunk is performed at a same time as moving the first live data of the second data chunk and the second live data of the third data chunk, wherein first dead data of the second data chunk and second dead data of the third data chunk are not moved with the moving of the second data chunk and the third data chunk.

2. The method of claim 1 , wherein the moving of the first data chunk comprises moving the first data chunk as-is while preserving an identification of the first data chunk at the second storage device.

3. The method of claim 2 , further comprising:
updating, by the processor, a first location information for the first live data of the second data chunk to point to the fourth data chunk in the second storage device; and
updating, by the processor a second location information for the second live data of the second data chunk to point to the fourth data chunk in the second storage device.

4. The method of claim 2 , further comprising, after the moving of the first data chunk from the first storage device, recovering, by the processor, a capacity of the first data chunk in the first storage device.

5. The method of claim 1 , wherein the moving the first data chunk comprises:
moving the first data chunk, coding data indicative of a location associated with the first data chunk, and a chunk identifier associated with the first data chunk.

6. The method of claim 5 , further comprising:
deleting, by the processor, the first data chunk from the first storage device, wherein the first data chunk is a storage block.

7. The method of claim 1 , further comprising:
prior to the moving of the first data chunk, determining, by the processor, that the first data chunk comprises immutable data.

8. The method of claim 1 , wherein the first data chunk comprises live data that is referenced.

9. The method of claim 1 , wherein the first storage device and the second storage device are geographically distributed devices.

10. The method of claim 1 , further comprising:
deleting, by the processor, the first dead data of the second data chunk and the second dead data of the third data chunk from the first storage device.

11. The method of claim 1 , wherein the first dead data of the second data chunk and the second dead data of the third data chunk are data that is not referenced.

12. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, cause the processor to perform operations, the operations comprising:
determining that a first use efficiency of a first data chunk satisfies a defined use efficiency threshold;
determining that a second use efficiency of a second data chunk and a third use efficiency of a third data chunk fail to satisfy the defined use efficiency threshold;
moving the first data chunk from a first storage device to a second storage device;
moving a first data segment from the second data chunk in the first storage device to a fourth data chunk in the second storage device; and
moving a second data segment from the third data chunk in the first storage device to the fourth data chunk in the second storage device, wherein the moving of the first data segment and the moving of the second data segment is performed during the moving the first data chunk.

13. The system of claim 12 , wherein the operations further comprise:
updating first information indicative of a first location of the first data segment and second information indicative of a second location of the second data segment to third information indicative of a third location of the second data chunk in the second storage device.

14. The system of claim 12 , wherein the operations further comprise:
after moving the first data segment, deleting the second data chunk from the first storage device; and
reacquiring a capacity of the second data chunk within the first storage device.

15. The system of claim 12 ,
wherein the first data segment and the second data segment comprise respective referenced data that is open for new writes.

16. The system of claim 12 , wherein the operations further comprise:
determining that a third use efficiency of a fourth data chunk fails to satisfy the defined use efficiency threshold, moving a third data segment from the fourth data chunk in the first storage device to the second data chunk in the second storage device during the moving the first data chunk, wherein the fourth data chunk comprises second referenced data that is open for new writes.

17. The system of claim 12 , wherein the operations further comprise:
prior to the moving of the first data chunk and the moving of the first data segment and the second data segment, determining that the first data chunk, the first data segment of the second data chunk, and the second data segment of the third data chunk comprise immutable data.

18. A non-transitory computer-readable medium comprising instructions that, when executed by a processor of a computing system, cause the processor to perform operations, the operations comprising:
determining a first data chunk is to be migrated from a first storage node device to a second storage node device, wherein the first data chunk is determined to comprise a first use efficiency level that satisfies a defined use efficiency level,
determining that a second data chunk comprises a first data segment that is to be migrated from the first storage node device to the second storage node device, wherein the second data chunk is determined to comprise a second use efficiency level that fails to satisfy the defined use efficiency level;
determining that a third data chunk comprises a second data segment that is to be migrated from the first storage node device to the second storage node device, wherein the third data chunk is determined to comprise a third use efficiency level that fails to satisfy the defined use efficiency level;
moving the first data chunk, the first data segment, and the second data segment from the first storage node device to the second storage node device; and
deleting a third data segment of the second data chunk, and a fourth data segment of the third data chunk from the first storage node device based on a determination that the third data segment and the fourth data segment do not comprise referenced data.

19. The non-transitory computer-readable medium of claim 18 , wherein the operations further comprise updating respective location information for the first data segment and the second data segment, and wherein the respective location information comprises a reference

Assignees

• Emc Ip Holding Co Llc

Inventors

• Danilov Mikhail
• Buinov Konstantin

Classifications

• G06F3/0649

  Lifecycle management · CPC title

• G06F3/0638

  Organizing or formatting or addressing of data · CPC title

• G06F3/0673

  Single storage device · CPC title

• G06F3/0604 Primary

  Improving or facilitating administration, e.g. storage management · CPC title

• G06F3/0608

  Saving storage space on storage systems · CPC title