Data storage system with adaptive file system over-provisioning

Patent metadata

Field	Value
Publication number	US-9864753-B1
Application number	US-201615085276-A
Country	US
Kind code	B1
Filing date	Mar 30, 2016
Priority date	Mar 30, 2016
Publication date	Jan 9, 2018
Grant date	Jan 9, 2018

Abstract

A data storage system tracks liability and insurance for an internal file system, liability being a number of slices needed to store file system data, insurance being a number of slices allowed to be consumed. A reserve of un-provisioned insurance is maintained from which slices are provisioned to the file system for use in storing file system data without immediate requirement for increasing the insurance. Slices are provisioned to the file system from the reserve of un-provisioned insurance based on a window criteria and a space criteria, the window criteria being that a number of free windows is less than a free window threshold, the space criteria being that a number of currently provisioned slices is less than a maximum allowed slices, which includes an overprovisioning factor applied to the primary file size to allow for growth of the primary file without immediate requirement for increasing provisioned insurance.

First claim

What is claimed is:

1. A method of operating a data storage system to manage use of pool storage by a file system of the data storage system, the pool storage provided by physical data storage devices and organized into slices each containing a predetermined number of contiguous blocks, the blocks of each slice further organized into windows of multiple contiguous blocks, the file system storing a primary file for a primary storage object and secondary files for secondary storage objects derived from the primary storage object, comprising:
continually tracking liability and insurance of the file system, the liability being a number of slices needed by the file system to store file system data, the insurance being a number of slices allowed to be consumed by the file system, the insurance including provisioned insurance and un-provisioned insurance, the provisioned insurance being a number of slices currently used by the file system, the un-provisioned insurance being a remainder portion not currently used by the file system;
maintaining a reserve of un-provisioned insurance from which slices are provisioned to the file system for use in storing file system data without immediate requirement for increasing the insurance, the reserve of un-provisioned insurance being maintained by increasing the insurance when a current value of the un-provisioned insurance drops below a predetermined non-zero low-insurance threshold; and
provisioning slices to the file system from the reserve of un-provisioned insurance, thereby increasing the provisioned insurance, based on both a window criteria and a space criteria, the window criteria being that a number of free windows is less than a predetermined non-zero free window threshold, a free window being a window whose blocks do not currently store file system data, the space criteria being that a number of currently provisioned slices is less than a maximum allowed slices, the maximum allowed slices including an overprovisioning factor applied to the primary file size to allow for growth of the primary file without immediate requirement for increasing the provisioned insurance.

2. The method of claim 1 , wherein the overprovisioning factor is a predetermined fraction of a current amount of free space in the pool storage, and wherein the maximum allowed slices further includes a factor for space consumed by the secondary storage objects as replicas of the primary storage object.

3. The method of claim 1 , wherein the pool storage is shared with one or more other file systems, and wherein increasing the insurance is conditioned on there being at least a predetermined minimum amount of free space in the pool to reduce the likelihood of starvation of the other file systems for storage space.

4. The method of claim 1 , further including:
tracking a number of evacuatable slices as a maximum of first and second values, the first value including a window value reflecting a current number of free windows beyond the free window threshold, the second value including an overprovisioning value reflecting a current amount of overprovisioning of slices to the file system; and
periodically performing a slice reclaim process to remove the number of evacuatable slices from the file system.

5. The method of claim 4 , wherein the first value further includes a hold-back value causing the number to be correspondingly smaller when the first value is greater than the second value, the smaller number resulting in retention of a corresponding number of slices in the file system that would otherwise be evacuatable, the retention avoiding an immediate need for returning slices to the file system upon completion of the slice reclaim process.

6. The method of claim 4 , wherein the slice reclaim process includes identifying particular slices to be evacuated using a selected one of two predetermined slice selection criteria, a first slice selection criteria being to select slices whose removal will cause the least amount of decrease in the number of free windows and being used when the first value is greater than the second value, a second slice selection criteria being to select slices in a decreasing-preference order and being used when the second value is greater than the first value, the decreasing-preference order including (1) empty data slices, (2) empty metadata slices, and (3) slices having the most free data.

7. The method of claim 6 , wherein selecting slices whose removal will cause the least amount of decrease in the number of free windows includes (1) computing a value X for each of a set of candidate slices, X being the sum of the number of free windows in a candidate slice and the number of free windows consumed by moving blocks out of the slice, and (2) preferentially selecting slices having lower values of X.

8. The method of claim 4 , further including, after the slice reclaim process has been performed, preventing removal of any additional slices from the file system for a predetermined timeout period.

9. The method of claim 1 , further including:
tracking conditions that establish start and stop triggers for a space maker process used to create un-fragmented free space in the file system;
beginning the space maker process upon satisfaction of conditions of a start trigger, execution of the space maker process including (1) selecting slices with windows to be evacuated, and (2) selecting windows within selected slices for evacuation, and (3) evacuating the selected windows of the selected slices; and
terminating the space maker process upon satisfaction of a stop trigger.

10. The method of claim 9 , wherein the start triggers include:
(1) a file system reclaim process is not executing, to avoid contention therewith;
(2) the file system has more than a predetermined number of provisioned data slices, to avoid executing the space maker process if the file system has just started to accept I/O;
(3) the free window count has dropped below a predetermined threshold that is greater than the free window threshold used for provisioning, to begin the space maker process before slices are over-provisioned to the file system; and
(4) there are sufficient partial windows whose evacuation will create sufficient free windows to bring a final free window count above a predetermined desired threshold.

11. The method of claim 9 , wherein the stop triggers are checked upon completion of processing of each slice and include:
(1) a free window count goes above the predetermined desired threshold; or
(2) there are insufficient partial windows remaining whose evacuation will create sufficient free windows to being the final free window count above the predetermined desired threshold.

12. The method of claim 9 , wherein the space maker process uses a slice selection criteria for selecting slices and a window selection criteria for selecting windows, the slice selection criteria including:
(1) the slice is not currently receiving I/O of sequential data writes, to avoid contention with production I/O; and
(2) a net number of free windows that can be created by evacuating the windows of the slice is above a created-windows threshold,
and the window selection criteria including that a number of free blocks in a window is above a freed-blocks threshold.

13. The method of claim 12 , further including dynamically varying the created-windows threshold during operation by:
if the number of slices selected for evacuation of windows is less than a predetermined fraction of total slices, then decreasing the created-windows threshold to increase aggressiveness of the slice selection; and
otherwise increasing the created-windows threshold to decrease the aggre

Assignees

• Emc Corp
• Emc Ip Holding Co Llc

Inventors

• Armangau Philippe
• Rashid Ahsan
• Nanda Kumari Bijayalaxmi
• Mathews Alexander

Classifications

• G06F3/0689

  Disk arrays, e.g. RAID, JBOD · CPC title

• G06F3/0653

  Monitoring storage devices or systems · CPC title

• G06F3/0608

  Saving storage space on storage systems · CPC title

• G06F17/30138 Primary

  Physics · mapped topic

• G06F3/0652

  Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket · CPC title