Adaptive namespace verification for single-node filesystems

Patent metadata

Field	Value
Publication number	US-12153546-B2
Application number	US-202217958862-A
Country	US
Kind code	B2
Filing date	Oct 3, 2022
Priority date	Oct 3, 2022
Publication date	Nov 26, 2024
Grant date	Nov 26, 2024

Abstract

Embodiments for providing adaptive namespace verification with high efficiency. The verification process is used in modern filesystems to check the integrity of key data structures, such for backup systems where providing thorough and efficient mechanisms for verification is critical. Embodiments accelerate the verification process by selectively applying continuous, differential, and integral verification techniques for startup and runtime repair purposes to overcome present problems for system capacities that exceed the petabyte range and impose significant time requirements using existing verification methods. Embodiments work with both single-node and distributed filesystems.

First claim

What is claimed is:

1. A method of verifying filesystem data in a single-node filesystem using an adaptive verification process, comprising:
maintaining metadata and file data;
tracking node events on a directory tree basis for the single node;
obtaining state information about the single node;
obtaining load information about the single-node, wherein the load information comprises specific client data traffic to the single node; and
dynamically determining, through the adaptive verification process selected from at least one of continuous verification, differential verification, and integral verification based on the tracked events, the state information and the load information.

2. The method of claim 1 further comprising periodically executing a snapshot process that provides a consistent point-in-time image between the metadata and corresponding file data.

3. The method of claim 2 wherein the continuous verification utilizes namespace operations including file creates, updates, and deletes logged into a transaction log and that are continuously verified using a verification thread operating on a verification cache.

4. The method of claim 3 wherein the differential verification comprises a periodic process distributed among the nodes and comprises comparing snapshot copies of a current state and a previous verified state of the filesystem using node assignments.

5. The method of claim 4 wherein the integral verification comprises taking an entire sweep of the filesystem metadata by comparing periodic snapshots of the filesystem on a node-by-node basis.

6. The method of claim 1 wherein the node events comprise at least one of: a node restart, a garbage collection completion, and normal operation.

7. The method of claim 6 wherein continuous verification is used for the node restart node event, integral verification is used for the garbage collection completion node event, and differential verification is used for the normal operation node event.

8. The method of claim 7 further comprising tracking additional events comprising system crashes, inbound load conditions seen by the node, and long-term aging of data.

9. The method of claim 8 further comprising implementing a defined policy in response to one or more of the additional events, and wherein the defined policy comprises adjusting verification process operational characteristics for the continuous verification, differential verification, and integral verification.

10. The method of claim 1 wherein the single-node filesystem comprises log-structured file system implemented in a Data Domain Filesystem, and wherein the namespace contains file metadata implemented as a B+ tree comprising a balanced tree of fixed depth, wherein a node is a page of space, and related metadata is stored in leaf pages.

11. A method of verifying filesystem data in a single-node filesystem using an adaptive verification process, comprising:
defining a rules driven knowledge base to implement a verification strategy based on defined node events and dynamic events;
periodically executing a snapshot process that provides a consistent point-in-time image between the metadata and corresponding file data; and
implementing, through the adaptive verification process, at least one of continuous verification, differential verification, and integral verification based on the defined node events and system events, wherein the continuous verification comprises a verification process utilizes namespace operations including file creates, updates, and deletes logged into a transaction log and that are continuously verified using a verification thread operating on a verification cache, the differential verification comprises a periodic process comparing snapshot copies of a current state and a previous verified state of the filesystem, and the integral verification comprises taking an entire sweep of the filesystem metadata by comparing periodic snapshots of the filesystem on the single-node.

12. The method of claim 11 wherein the node events comprise at least one of: a node restart, a garbage collection completion, and normal operation, and wherein continuous verification is used for the node restart, integral verification is used for the garbage collection completion event, and differential verification is used for the normal operation event.

13. The method of claim 12 wherein the dynamic events comprise system crashes, inbound load conditions seen by the load node, and long-term aging of data.

14. The method of claim 13 further comprising implementing a policy in response to one or more of the dynamic events, and wherein the policies comprise adjusting verification process operational characteristics for the continuous verification, differential verification, and integral verification.

15. The method of claim 11 wherein the single-node filesystem comprises a node having a log-structured file system implemented in a Data Domain Filesystem, and wherein the namespace contains file metadata implemented as a B+ tree comprising a balanced tree of fixed depth, wherein a node is a page of space, and related metadata is stored in leaf pages, and further wherein the single-node filesystem separately maintains the metadata and file data.

16. A method of verifying filesystem data in a single-node filesystem using an adaptive verification process, comprising:
defining a rules driven knowledge base to implement a verification strategy based on defined node events and dynamic events;
periodically executing a snapshot process that provides a consistent point-in-time image between the metadata and corresponding file data; and
implementing, through the adaptive verification process, at least one of continuous verification, differential verification, and integral verification based on the defined node events and system events, wherein the node events comprise at least one of: a node restart, a garbage collection completion, and normal operation, and wherein the dynamic events comprise system crashes, inbound load conditions seen by the node, and long-term aging of data.

17. The method of claim 16 wherein continuous verification is used for the node restart event, integral verification is used for the garbage collection completion event, and differential verification is used for the normal operation event.

18. The method of claim 17 further comprising implementing a policy in response to one or more of the dynamic events, and wherein the policies comprise adjusting verification process operational characteristics for the continuous verification, differential verification, and integral verification.

19. The method of claim 18 wherein:
the continuous verification comprises a verification process utilizes namespace operations including file creates, updates, and deletes logged into a transaction log and that are continuously verified using a verification thread operating on a verification cache,
the differential verification comprises a periodic process comparing snapshot copies of a current state and a previous verified state of the filesystem, and
the integral verification comprises taking an entire sweep of the filesystem metadata by comparing periodic snapshots of the filesystem on a node-by-node basis.

20. The method of claim 16 wherein the node filesystem comprises a node having a log-structured file system implemented in a Data Domain Filesystem, and wherein the namespace contains file metadata implemented as a B+ tree comprising a balanced tree of fixed depth, wherein a node is a page of space, a

Assignees

• Dell Products Lp

Inventors

• Hickey Charles J
• Lakshmikantha Shashikanth
• Mamidi Murthy V

Classifications

• G06F16/162

  Delete operations (erasing in storage systems G06F3/0652) · CPC title

• G06F16/172

  Caching, prefetching or hoarding of files · CPC title

• G06F16/1734 Primary

  Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs · CPC title