Online replacement of physical storage in a virtual storage system

Patent metadata

Field	Value
Publication number	US-9250823-B1
Application number	US-201414282318-A
Country	US
Kind code	B1
Filing date	May 20, 2014
Priority date	May 20, 2014
Publication date	Feb 2, 2016
Grant date	Feb 2, 2016

Abstract

Physical storage is replaced online in a data integrity storage system comprising RAID groups of physical disks in separate enclosures (shelves). All disks of a RAID group are located on a corresponding shelf, and each shelf is mapped by an internal file system to a separate independent logical address space partitioned into a plurality of blocks forming a blockset containing data and metadata. Source shelf disk data is moved online to disks of a target shelf using invulnerable data movement that ensures the integrity of the data, and source shelf blockset metadata is migrated to a corresponding target shelf blockset. After verifying the correctness of the target data and metadata, the source shelf and blockset are removed.

First claim

The invention claimed is:

1. A method of replacing physical storage online in a storage system having an internal file system, comprising:
mapping, by said internal file system, a plurality of physical storage disks of a storage shelf to a separate independent logical address space partitioned into a plurality of blocks forming a blockset, each of said blocks containing data and block metadata, said blockset corresponding to a virtual storage device;
migrating a source shelf while said storage system is online by moving data from the physical storage disks of the source shelf and from source blocks of a corresponding source blockset to storage disks of a target shelf and to target blocks of a corresponding target blockset, respectively;
updating block metadata of said target blocks to reference said corresponding target blockset; and
committing said migrating by removing said source shelf from said storage system and said source blockset from said internal file system.

2. The method of claim 1 further comprising verifying the correctness of the data moved to the target shelf and target blockset prior to said removing said source shelf and source blockset.

3. The method of claim 2 , wherein said verifying comprises reading data from source disks of the source shelf and writing said data into a buffer memory, calculating a first checksum of said data as read, further writing the data in said buffer memory to target disks of said target shelf, calculating a second checksum of the data written to the target disks, and comparing said first and second checksums to verify the data written to the target disks.

4. The method of claim 3 , wherein said source disks are part of a RAID group, and the method further comprises comparing said first checksum with parity information of said RAID group to verify the correctness of the data written to the buffer memory.

5. The method of claim 1 , wherein said mapping comprises mapping all of the storage disks of the source shelf and the target shelf to a corresponding RAID group and to a corresponding logical address space of each shelf.

6. The method of claim 5 further comprising executing a pre-migration check prior to said migrating to verify that none of the disks of said each shelf span a physical boundary of said each shelf.

7. The method of claim 1 further comprising creating a target disk group of all of the storage disks of said target shelf, exposing said target disk group to said file system, and creating by said file system said target blockset for said target disk group.

8. The method of claim 1 , wherein said migrating comprises migrating data from source blocks of said source blockset to corresponding target blocks of said target blockset, and said updating block metadata comprises updating said block metadata of said target blocks to reference a mapping of said each target block to an offset in a logical address space of said target blockset.

9. The method of claim 8 , wherein said migrating data from said source blocks comprising moving data only from allocated source blocks to free target blocks.

10. The method of claim 1 , wherein said removing comprises removing the source shelf after moving all data from said source shelf to said target shelf and verifying the correctness of the moved data.

11. The method of claim 1 , wherein said block metadata includes a block identifier and an offset of the block in a corresponding blockset.

12. The method of claim 1 , wherein each blockset has a header comprising blockset metadata that includes information identifying each block of said blockset and an offset of the block in a corresponding virtual storage device.

13. Non-transitory computer readable media storing instructions for controlling the operation of a computer to perform a method of replacing physical storage online in a storage system having an internal file system, comprising instructions for:
mapping a plurality of physical storage disks of a storage shelf to a separate independent logical address space partitioned into a plurality of blocks forming a blockset, each of said blocks containing data and block metadata, said blockset corresponding to a virtual storage device;
migrating a source shelf while said storage system is online by moving data from the physical storage disks of the source shelf and from source blocks of a corresponding source blockset to storage disks of a target shelf and to target blocks of a corresponding target blockset, respectively;
updating block metadata of said target blocks to reference said corresponding target blockset; and
committing said migrating by removing said source blockset and a corresponding virtual storage device from said internal file system.

14. Non-transitory computer readable media of claim 13 , wherein said instructions for mapping comprise instructions for mapping all of the storage disks of the source shelf and the target shelf to a corresponding RAID group and to a corresponding logical address space of each shelf.

15. Non-transitory computer readable media of claim 13 further comprising instructions for executing a pre-migration check prior to said migrating to verify that none of the disks of said each shelf span a physical boundary of said each shelf.

16. Non-transitory computer readable media of claim 13 further comprising instructions for creating a target disk group of all of the storage disks of said target shelf, exposing said target disk group to said file system, and creating by said file system said target blockset for said target disk group.

17. Non-transitory computer readable media of claim 13 , wherein said instructions for migrating comprise instructions for migrating data from source blocks of said source blockset to corresponding target blocks of said target blockset, and instructions for updating said block metadata of said target blocks to reference a mapping of said each target block to an offset in a logical address space of said target blockset.

Assignees

• Emc Corp

Inventors

• KAMAT Mahesh
• Liang Shuang

Classifications

• G06F3/0689

  Disk arrays, e.g. RAID, JBOD · CPC title

• G06F3/0643

  Management of files · CPC title

• G06F3/0647 Primary

  Migration mechanisms · CPC title

• G06F3/0619

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F3/0632

  by initialisation or re-initialisation of storage systems · CPC title