Grouping data to conserve storage capacity

Patent metadata

Field	Value
Publication number	US-12530318-B2
Application number	US-202118035137-A
Country	US
Kind code	B2
Filing date	Jan 12, 2021
Priority date	Jan 12, 2021
Publication date	Jan 20, 2026
Grant date	Jan 20, 2026

Abstract

In some examples, a computing device stores root metadata data structures (DSs) together in a group root metadata filesystem block, and stores, in a data filesystem block, at least first file data referenced by the first root metadata DS and second file data referenced by a second root metadata DS. The computing device may determine a reference count of the data filesystem block based on the first file data referenced by the first root metadata DS and the second file data referenced by the second root metadata DS. In addition, the computing device may determine a third file data having content that matches the first file data and referenced by a third root metadata DS. The computing device may update the third root metadata DS to reference the first file data, and update the reference count of the data filesystem block based on updating the third root metadata DS.

First claim

The invention claimed is:

1 . A system configured to conserve storage capacity, the system comprising:
at least one computing device including one or more processors configured to perform operations for storing file data and metadata for the file data, the metadata including root metadata data structures (DSs), the operations comprising:
storing a plurality of root metadata DSs together in a group root metadata DS filesystem block of a filesystem, the plurality of root metadata DSs including a first root metadata DS;
determining that a first file including first file data referenced by the first root metadata DS and a second file including second file data referenced by a second root metadata DS are smaller than a threshold file size;
based on the first file and the second file being smaller than the threshold size, storing, in a same first data filesystem block of the filesystem, at least the first file data referenced by the first root metadata DS and the second file data referenced by a second root metadata DS, the second root metadata DS stored in the group root metadata DS or another group root metadata DS;
determining a reference count of the first data filesystem block based at least on the first file data referenced by the first root metadata DS and the second file data referenced by the second root metadata DS, wherein determining the reference count includes accessing a bitmap corresponding to at least a portion of the filesystem, the bitmap including a plurality of sets of multiple bits, wherein a respective set of multiple bits corresponds to a respective data file system block, wherein the respective set of multiple bits indicates how many data files smaller than the threshold file size are stored in the corresponding respective data filesystem block;
determining a third file data having content that matches content of the first file data, the third file data referenced by a third root metadata DS;
performing deduplication by updating the third root metadata DS to reference the first file data instead of the third file data; and
updating the reference count of the first data filesystem block based on updating the third root metadata DS.

2 . The system as recited in claim 1 , the operations further comprising:
when storing a new root metadata DS, determining whether the group root metadata DS filesystem block has sufficient storage space for the new root metadata DS; when the group root metadata DS filesystem block has sufficient storage space, storing the new root metadata DS in the group root metadata DS filesystem block; and when the group root metadata DS filesystem block has insufficient storage space, storing the new root metadata DS in a new group root metadata DS filesystem block.

3 . The system as recited in claim 2 , wherein the operation of determining whether the group root metadata DS filesystem block has sufficient storage space for the new root metadata DS comprises accessing the bitmap to determine from a set of multiple bits corresponding to the group root metadata DS filesystem block, a reference count for the corresponding group root metadata DS filesystem block.

4 . The system as recited in claim 1 , wherein the operations are automatically performed in response to a write request.

5 . The system as recited in claim 1 , wherein the first root metadata DS and the second root metadata DS are included in the same group root metadata DS filesystem block.

6 . The system as recited in claim 1 , wherein, when another root metadata DS in another root metadata DS filesystem block references a file data in the first file data filesystem block, the reference count of the first file data filesystem block is incremented in the bitmap.

7 . The system as recited in claim 1 , further comprising storing intermediate metadata DSs with other intermediate metadata DSs in a group intermediate metadata DS filesystem block.

8 . The system as recited in claim 7 , the operations further comprising:
when storing a new intermediate metadata DS, determining whether the group intermediate metadata DS filesystem block has sufficient storage space for the intermediate metadata DS; when the group intermediate metadata DS filesystem block has sufficient storage space, storing the intermediate metadata DS in the existing group intermediate metadata DS filesystem block; and when the group intermediate metadata DS filesystem block has insufficient storage space, storing the intermediate metadata DS in a new group intermediate metadata DS filesystem block.

9 . The system as recited in claim 1 , wherein a first intermediate metadata DS in the group intermediate metadata DS filesystem block references a second intermediate metadata DS in the group intermediate metadata DS filesystem block.

10 . The system as recited in claim 9 , wherein the first intermediate metadata DS is referenced by the respective root metadata DS in the group root metadata DS file system block.

11 . The system as recited in claim 10 , wherein the second intermediate metadata DS references a data file system block.

12 . A method for conserving storage capacity, the method comprising:
storing, by a computing device, a plurality of root metadata DSs together in a group root metadata DS filesystem block of a filesystem, the plurality of root metadata DSs including a first root metadata DS; determining that a first file including first file data referenced by the first root metadata DS and a second file including second file data referenced by a second root metadata DS are smaller than a threshold file size; based on the first file and the second file being smaller than the threshold size, storing, in a same first data filesystem block of the filesystem, at least the first file data referenced by the first root metadata DS and the second file data referenced by a second root metadata DS, the second root metadata DS stored in the group root metadata DS or another group root metadata DS; determining a reference count of the first data filesystem block based at least on the first file data referenced by the first root metadata DS and the second file data referenced by the second root metadata DS, wherein determining the reference count includes accessing a bitmap corresponding to at least a portion of the filesystem, the bitmap including a plurality of sets of multiple bits, wherein a respective set of multiple bits corresponds to a respective data file system block, wherein the respective set of multiple bits indicates how many data files smaller than the threshold file size are stored in the corresponding respective data filesystem block; determining a third file data having content that matches content of the first file data, the third file data referenced by a third root metadata DS; performing deduplication by updating the third root metadata DS to reference the first file data; and updating the reference count of the first data filesystem block based on updating the third root metadata DS.

13 . The method as recited in claim 12 , further comprising:
when storing a new root metadata DS, determining whether the group root metadata DS filesystem block has sufficient storage space for the new root metadata DS; when the group root metadata DS filesystem block has sufficient storage space, storing the new root metadata DS in the group root metadata DS filesystem block; and when the group root metadata DS filesystem block has insufficient storage space, storing the new root metadata DS in a new group root metadata DS filesystem block.

14 . The method as recited in claim 12 , wherein the operations are automatically performed in re

Assignees

• Hitachi Vantara Llc

Inventors

• Smith Jonathan
• Leverington Mark
• Crosland Simon

Classifications

• G06F16/1748

  De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641) · CPC title

• G06F16/164

  File meta data generation · CPC title

• G06F16/1727 Primary

  Details of free space management performed by the file system (saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064) · CPC title