Data storage system using in-memory structure for reclaiming space from internal file system to pool storage

Patent metadata

Field	Value
Publication number	US-10936195-B2
Application number	US-201715797323-A
Country	US
Kind code	B2
Filing date	Oct 30, 2017
Priority date	Oct 30, 2017
Publication date	Mar 2, 2021
Grant date	Mar 2, 2021

Abstract

A data storage system manages use of a pool of secondary storage by internal file systems hosting logical data storage objects accessed by clients. A choose-and-mark-slices operation scans the file system to identify a subset of the slices as evacuatable slices to be de-provisioned and returned to the pool, the subset having a size exceeding a per-iteration limit observed by a separate evacuate-slices operation. The subset is recorded in an in-memory structure. Each iteration of the evacuate-slices operation (1) examines the in-memory structure to select slices for evacuation, no more than the per-iteration limit being selected, (2) moves data from the selected slices to the remaining slices of the set, and (3) subsequently (i) updates metadata of the file system to reflect the moving of the data and (ii) de-provisions the selected slices from the file-system to return them to the pool for subsequent re-provisioning.

First claim

What is claimed is:

1. A method of operating a data storage system to manage use of a pool of secondary storage by internal file systems hosting respective logical data storage objects accessed by clients of the data storage system, the pool being organized into fixed-size slices dynamically provisioned to and de-provisioned from the file systems, the method comprising, for each file system:
performing a choose-and-mark-slices operation that includes scanning a set of slices of the file system to identify, based on data occupancy of the slices, a subset of the slices as evacuatable slices to be de-provisioned and returned to the pool, the subset of slices storing data to be moved to remaining slices of the set by iterations of a separate evacuate-slices operation, the subset having a size exceeding a predetermined fixed per-iteration limit observed by the evacuate-slices operation in each iteration thereof, the slices of the subset being recorded in an in-memory structure to communicate an identification of the subset to the evacuate-slices operation; and
regularly performing the iterations of the evacuate-slices operation, each iteration (1) examining the in-memory structure to select slices for evacuation from among the evacuatable slices, with no more than the per-iteration limit of occupied slices being selected, (2) moving data from the selected slices to the remaining slices of the set to thereby empty the selected slices, and (3) subsequently (i) updating metadata of the file system to reflect the moving of the data and (ii) de-provisioning the selected slices from the file-system to return them to the pool for subsequent re-provisioning to the file systems,
wherein the choose-and-mark-slices operation and each iteration of the evacuate-slices operation are independently scheduled such that (1) the choose-and-mark-slices operation identifies the subset of slices and records the subset in the in-memory structure in a single pass notwithstanding that the size of the subset exceeds the per-iteration limit of the evacuate-slices operation, (2) a first iteration of the evacuate-slices operation selects and operates on a first group of the slices of the subset recorded in the in-memory structure, the first group having a size equal to the per-iteration limit, and (3) a subsequent second iteration of the evacuate-slices operation selects and operates upon other slices of the subset remaining after the first iteration of the evacuate-slices operation.

2. The method of claim 1 , wherein each of the slices includes a predetermined number of fixed-size data blocks, and the choose-and-mark-slices operation and evacuate-slices operation use an intermediate multiple-block segment as a data unit in evaluating occupancy and moving data from a selected slice to a remaining slice.

3. The method of claim 1 , wherein operation of the data storage system further includes regular file operations of the file system including allocating provisioned slices to store newly written file data, and wherein the allocating of provisioned slices avoids using slices marked for evacuation as recorded in the in-memory structure.

4. The method of claim 1 , wherein operation of the data storage system further includes a virtual pooling process regularly performing relocation of provisioned slices within the file system, the virtual pooling process being unaware of slice marking for evacuation, and the per-iteration limit is selected to limit duplicate data movement occurring when the virtual pooling process relocates slices marked for evacuation.

5. The method of claim 1 , wherein examining the in-memory structure to select slices for evacuation results in selecting the per-iteration limit of occupied slices as well as additional slices that are fully unoccupied and thus require no data movement for evacuation.

6. The method of claim 1 , wherein the choose-and-mark-slices operation identifies slices for evacuation according to decreasing data occupancy so that more-occupied slices are evacuated before less-occupied slices.

7. The method of claim 6 , wherein the slices are identified using an ordered list.

8. The method of claim 1 , wherein the choose-and-mark-slices operation is triggered on a timing basis to be performed periodically.

9. The method of claim 1 , wherein the choose-and-mark-slices operation is triggered on an event basis to be performed in response to occurrence of an operational event.

10. The method of claim 9 , wherein the operational event is one or more of (1) deletion of a logical object stored as a file in the file system, and (2) a size of a map-for-deallocate reaches a predefined threshold.

11. A data storage system, comprising:
data storage devices providing media for secondary storage;
interface circuitry coupling the data storage system to the data storage devices and to remote data storage clients; and
processing circuitry storing and executing computer program instructions to cause the data storage system to perform a method of managing use of a pool of secondary storage by internal file systems hosting respective logical data storage objects accessed by data storage clients, the pool being drawn from the media of the data storage devices and being organized into fixed-size slices dynamically provisioned to and de-provisioned from the file systems, the method including:
performing a choose-and-mark-slices operation that includes scanning a set of slices of the file system to identify, based on data occupancy of the slices, a subset of the slices as evacuatable slices to be de-provisioned and returned to the pool, the subset of slices storing data to be moved to remaining slices of the set by iterations of a separate evacuate-slices operation, the subset having a size exceeding a predetermined fixed per-iteration limit observed by the evacuate-slices operation in each iteration thereof, the slices of the subset being recorded in an in-memory structure to communicate an identification of the subset to the evacuate-slices operation; and
regularly performing the iterations of the evacuate-slices operation, each iteration (1) examining the in-memory structure to select slices for evacuation from among the evacuatable slices, with no more than the per-iteration limit of occupied slices being selected, (2) moving data from the selected slices to the remaining slices of the set to thereby empty the selected slices, and (3) subsequently (i) updating metadata of the file system to reflect the moving of the data and (ii) de-provisioning the selected slices from the file-system to return them to the pool for subsequent re-provisioning to the file systems,
wherein the choose-and-mark-slices operation and each iteration of the evacuate-slices operation are independently scheduled such that (1) the choose-and-mark-slices operation identifies the subset of slices and records the subset in the in-memory structure in a single pass notwithstanding that the size of the subset exceeds the per-iteration limit of the evacuate-slices operation, (2) a first iteration of the evacuate-slices operation selects and operates on a first group of the slices of the subset recorded in the in-memory structure, the first group having a size equal to the per-iteration limit, and (3) a subsequent second iteration of the evacuate-slices operation selects and operates upon other slices of the subset remaining after the first iteration of the evacuate-slices operation.

12. The data storage system of claim 11 , wherein each of the slices includes a predetermined number of fixed-size data blocks, and the choose-and-mark-slices operation and evacuate-slices operation use an intermediate multiple-block segment as a data unit in evalu

Assignees

• Emc Ip Holding Co Llc

Inventors

• Liu Xianlong
• Yu Changyong
• Dou Ruiling
• Mathews Alexander S

Classifications

• G06F3/0652

  Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket · CPC title

• G06F3/0631

  by allocating resources to storage systems · CPC title

• G06F3/0608 Primary

  Saving storage space on storage systems · CPC title

• G06F3/0689

  Disk arrays, e.g. RAID, JBOD · CPC title

• G06F3/0665

  at area level, e.g. provisioning of virtual or logical volumes · CPC title