Continuous namespace verification for single-node filesystems

Patent metadata

Field	Value
Publication number	US-12314226-B2
Application number	US-202217958751-A
Country	US
Kind code	B2
Filing date	Oct 3, 2022
Priority date	Oct 3, 2022
Publication date	May 27, 2025
Grant date	May 27, 2025

Abstract

Embodiments for providing adaptive namespace verification with high efficiency. The verification process is used in modern filesystems to check the integrity of key data structures, such for backup systems where providing thorough and efficient mechanisms for verification is critical. Embodiments accelerate the verification process by selectively applying continuous, differential, and integral verification techniques for startup and runtime repair purposes to overcome present problems for system capacities that exceed the petabyte range and impose significant time requirements using existing verification methods. Embodiments work with both single-node and distributed filesystems.

First claim

What is claimed is:

1. A method of verifying filesystem data in a single-node system using a continuous verification process, comprising:
checking, upon a system restart after an event, metadata just written to persistent storage by monitoring the flow of transactions written to the persistent storage;
snooping namespace writes that are logged in a transaction log configured to store all namespace operations;
storing the logged namespace writes in a verification cache;
receiving a list of modified pages replayed after the system restart; and
verifying, using an idle thread, each modified page by backtracking through events of the event to ensure that replayed pages have been correctly written to the persistent storage.

2. The method of claim 1 wherein the namespace stores names of directories and files, and related information as metadata that includes file ownership, creation time, and wherein the namespace operations comprise filesystem creates, updates, and deletes.

3. The method of claim 2 wherein the namespace of the single node contains file metadata implemented as a B+ tree comprising a balanced tree of fixed depth, wherein the node is a page of space and all metadata is stored in leaf pages.

4. The method of claim 3 wherein the verification cache is embodied in a B+ tree and configured to hold a condensed cache of metadata modifications, and has a key/value orientation.

5. The method of claim 1 wherein the transaction log comprises a persistent transaction log comprising a circular buffer data structure having a fixed size.

6. The method of claim 5 further comprising:
translating the transaction log entries are into page identifiers that are affected by the change;
condensing page information into a stream of identifiers; and
storing, in batches upon a flush operation of the transaction log, the stream in the verification cache.

7. The method of claim 6 wherein the keys comprise unique identifiers of the Mtree B+ Tree pages (DMBT pages) and inode objects.

8. The method of claim 7 wherein the keys are provided in a format: <mid><sid><page_no>, and <pid><cid><sid> respectively.

9. The method of claim 8 wherein information regarding modified files and their connection to directories and specific to the continuous verification process is saved in a value portion of key-value structures in a key-value (KV) store.

10. The method of claim 1 wherein the idle thread comprises a thread that is idle when there is no work for it to consume and that is activated when one of: new keys are added to the verification cache, or when a set timer count has expired.

11. A single-node system for verifying filesystem data in a single-node system using a continuous verification process, comprising:
a non-transitory, hardware-based persistent storage storing metadata written to the filesystem;
a monitor component monitoring the flow of transactions written to the persistent storage to check, upon a system restart after an event, metadata just written to the persistent;
a transaction log storing all namespace operations;
a processor-based component snooping namespace writes that are logged in the transaction log;
a verification cache storing the logged namespace writes; and
an idle thread processing component receiving a list of modified pages replayed after the system restart, and verifying, using, each modified page by backtracking through events of the event to ensure that replayed pages have been correctly written to the persistent storage.

12. The system of claim 11 wherein the namespace stores names of directories and files, and related information as metadata that includes file ownership, creation time, and wherein the namespace operations comprise filesystem creates, updates, and deletes, and further wherein the namespace of the single node contains file metadata implemented as a B+ tree comprising a balanced tree of fixed depth, wherein the node is a page of space and all metadata is stored in leaf pages, and yet further wherein the verification cache is embodied in a B+ tree and configured to hold a condensed cache of metadata modifications, and has a key/value orientation.

13. The system of claim 11 wherein the transaction log comprises a persistent transaction log comprising a circular buffer data structure having a fixed size.

14. The system of claim 13 wherein the idle thread comprises a thread that is idle when there is no work for the thread to consume and that is activated when one of: new keys are added to the verification cache, or when a set timer count has expired.

Assignees

• Dell Products Lp

Inventors

• Lakshmikantha Shashikanth
• Hickey Charles J
• Mamidi Murthy V

Classifications

• G06F16/1734 Primary

  Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs · CPC title