Data storage system using paced deallocation of truncated file blocks

Patent metadata

Field	Value
Publication number	US-10146780-B1
Application number	US-201615078672-A
Country	US
Kind code	B1
Filing date	Mar 23, 2016
Priority date	Mar 23, 2016
Publication date	Dec 4, 2018
Grant date	Dec 4, 2018

Abstract

In a data storage system an internal file system stores client storage objects as respective files. A count of actual free space includes blocks available for allocation to store new file data, and a count of reported free space includes the actual free space as well as blocks of recently deleted file data undergoing deallocation. The reported free space count is used to initiate file system operations requiring allocation of free space, while the actual free space count is used to continue initiated file system operations based on availability of actual free space. Upon a truncation, a reporting update estimates or counts a number of blocks to be freed and adds the number to the reported free space count, and a background update is initiated to deallocate truncated blocks at a paced rate and update the actual free space count as blocks are deallocated.

First claim

What is claimed is:

1. A method of operating a data storage system, comprising:
operating an internal file system using physical storage devices of the data storage system for underlying data storage, the internal file system storing client storage objects as respective files of the file system, the file system including allocated space occupied by existing file data and free space available for storing new file data;
maintaining respective counts of actual free space and reported free space of the file system, the actual free space including blocks currently available for allocation to store new file data, the reported free space including the actual free space as well as blocks of recently deleted file data undergoing deallocation, the count of reported free space being used to initiate file system operations requiring allocation of free space, the count of actual free space being used to continue initiated file system operations based on availability of actual free space; and
upon truncation of a file, performing a reporting update and initiating a background update, the reporting update either estimating or counting a number of blocks to be freed by the truncation and adding the number to the reported free space for use in initiating subsequent file system operations requiring allocation of free space, the background update deallocating truncated blocks of the file and updating the count of actual free space as the truncated blocks are deallocated, the background update being performed at a paced rate permitting initiation of new file system operations before all the truncated blocks have been deallocated.

2. The method of claim 1 , further including, during an initiated file system operation requiring allocation of an amount of free space greater than the actual free space, (i) temporarily suspending the file system operation and de-pacing the background update to hasten the deallocation of the truncated blocks, and (ii) upon the count of actual free space reaching a desired higher value by the deallocation of the truncated blocks, resuming the file system operation.

3. The method of claim 1 , wherein the file system includes files storing snapshots of the file being truncated, and wherein estimating a number of blocks to be freed by the truncation includes evaluating metadata of the file to determine the number of file system blocks that are uniquely allocated to the file, indicating that such file system blocks are not shared with any of the snapshots and thus can be deallocated.

4. The method of claim 3 , wherein the file being truncated stores a primary storage object and the files storing snapshots store snapshot images of the primary storage object.

5. The method of claim 3 , wherein the file being truncated is a file storing a snapshot image of a primary storage object stored in another file.

6. The method of claim 1 , wherein the truncation operation is performed in connection with deletion of the file, the truncation operation truncating the file to a size of zero.

7. The method of claim 1 , wherein a heuristic is used for pacing the background update, the heuristic employing factors selected from CPU utilization, I/O latency, and number of client I/Os.

8. The method of claim 1 , wherein initiating the background update includes:
during a first time period, adjusting the paced rate, which permits initiation of new file system operations before all of the truncated blocks have been deallocated, to a first selected rate of deallocating truncated blocks of the file, and
during a second time period after the first time period, adjusting the paced rate, which permits initiation of new file system operations before all of the truncated blocks have been deallocated, to a second selected rate of deallocating truncated blocks of the file, the second selected rate being faster than the first selected rate to accelerate deallocation of truncated blocks of the file in response to an increase in demand for free space.

9. A data storage system, comprising:
physical storage devices;
respective interfaces to the physical storage devices and to a network via which client storage requests are received; and
one or more storage processors coupled to the physical storage devices and to the network via the respective interfaces, the storage processors configured and operative to perform a method including:
operating an internal file system using the physical storage devices for underlying data storage, the internal file system storing client storage objects as respective files of the file system, the file system including allocated space occupied by existing file data and free space available for storing new file data;
maintaining respective counts of actual free space and reported free space of the file system, the actual free space including blocks currently available for allocation to store new file data, the reported free space including the actual free space as well as blocks of recently deleted file data undergoing deallocation, the count of reported free space being used to initiate file system operations requiring allocation of free space, the count of actual free space being used to continue initiated file system operations based on availability of actual free space; and
upon truncation of a file, performing a reporting update and initiating a background update, the reporting update either estimating or counting a number of blocks to be freed by the truncation and adding the number to the reported free space for use in initiating subsequent file system operations requiring allocation of free space, the background update deallocating truncated blocks of the file and updating the count of actual free space as the truncated blocks are deallocated, the background update being performed at a paced rate permitting initiation of new file system operations before all the truncated blocks have been deallocated.

10. The data storage system of claim 9 , wherein the method further includes, during an initiated file system operation requiring allocation of an amount of free space greater than the actual free space, (i) temporarily suspending the file system operation and de-pacing the background update to hasten the deallocation of the truncated blocks, and (ii) upon the count of actual free space reaching a desired higher value by the deallocation of the truncated blocks, resuming the file system operation.

11. The data storage system of claim 9 , wherein the file system includes files storing snapshots of the file being truncated, and wherein estimating a number of blocks to be freed by the truncation includes evaluating metadata of the file to determine the number of file system blocks that are uniquely allocated to the file, indicating that such file system blocks are not shared with any of the snapshots and thus can be deallocated.

12. The data storage system of claim 11 , wherein the file being truncated stores a primary storage object and the files storing snapshots store snapshot images of the primary storage object.

13. The data storage system of claim 11 , wherein the file being truncated is a file storing a snapshot image of a primary storage object stored in another file.

14. The data storage system of claim 9 , wherein the truncation operation is performed in connection with deletion of the file, the truncation operation truncating the file to a size of zero.

15. The data storage system of claim 9 , wherein a heuristic is used for pacing the background update, the heuristic employing factors selected from CPU utilization, I/O latency, and number of client I/Os.

16. A non-transitory computer-readable m

Assignees

• Emc Corp
• Emc Ip Holding Co Llc

Inventors

• Bassov Ivan

Classifications

• G06F16/122 Primary

  using management policies (point-in-time backing up or restoration of persistent data G06F11/1446; file migration policies for HSM systems G06F16/185) · CPC title

• G06F16/1727

  Details of free space management performed by the file system (saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064) · CPC title

• G06F17/30082 Primary

  Physics · mapped topic

• G06F17/30138

  Physics · mapped topic