Write-ahead log maintenance and recovery

Patent metadata

Field	Value
Publication number	US-11449239-B2
Application number	US-202017001294-A
Country	US
Kind code	B2
Filing date	Aug 24, 2020
Priority date	Apr 18, 2016
Publication date	Sep 20, 2022
Grant date	Sep 20, 2022

Abstract

A system for tracking metadata changes and recovering from system interruptions. With host I/O, corresponding metadata incremental changes are aggregated and stored in a write-ahead log before being performed to their in-memory buffers. As those buffers are flushed, checkpoints are created and stored in the log. As the log wraps to the start, older entries are overwritten after they are freed from any remaining dependencies by newer checkpoints. If metadata entities have not created new checkpoints, they are instructed to in order to free up space for new aggregated batches and checkpoints. After an interruption, the wrap point is located in the log. From the wrap point, the log is scanned backwards to provide checkpoints to metadata entities. The log is then scanned forwards to perform changes specified by aggregated batches. The metadata entities' volatile memory states are recovered to what they were before the interruption.

First claim

What is claimed is:

1. A method, comprising:
writing, by a storage controller, an incremental change set in a write ahead log comprising a plurality of incremental change sets in a non-volatile memory identified by a wrap point, the incremental change set corresponding to a condition of metadata in volatile memory for a metadata client, the incremental change set being written to the non-volatile memory prior to an update to the metadata;
tracking a checkpoint stored in the write ahead log to at least one of the plurality of incremental change sets, an oldest incremental change set corresponding to a region beyond the wrap point; and
determining, from the tracking, that the checkpoint in the write ahead log records a dependency of the metadata client to the oldest incremental change set, a new checkpoint being persisted in the write ahead log for the metadata client.

2. The method of claim 1 , wherein the write ahead log is stored in a solid state drive storing the metadata client.

3. The method of claim 1 , wherein determining that the checkpoint in the write ahead log records the dependency of the metadata client to the oldest incremental change set comprises:
accessing, by the storage controller, a list maintained for the region that identifies a plurality of metadata clients, including the metadata client, that maintain the checkpoint with the dependency to the oldest incremental change set.

4. The method of claim 1 , further comprising:
overwriting the oldest incremental change set subsequent to persisting the new checkpoint.

5. The method of claim 4 , further comprising:
maintaining a sequence number in the write ahead log, wherein the sequence number corresponds to a number of times that the wrap point has wrapped from an end of an address range to a start of the address range for the write ahead log; and
incrementing the sequence number in a block descriptor for the new checkpoint in response to the wrap point progressing to the start of the address range from the end of the address range.

6. The method of claim 1 , wherein the write ahead log comprises a circular buffer that wraps from an ending logical block address (LBA) to a starting LBA, the wrap point being between the ending LBA and the starting LBA and corresponding to a most recent dependency in the write ahead log.

7. The method of claim 1 , wherein the condition of metadata is either current or having stale components.

8. The method of claim 1 , further comprising:
persisting the new checkpoint in response to an amount of free space in the write ahead log falling below a threshold.

9. The method of claim 1 , wherein an updated dependency of the new checkpoint corresponds to a subsequent incremental change set that has a first dependency number being greater than a second dependency number of the oldest incremental change set.

10. A non-transitory machine readable medium having stored thereon instructions for performing a method comprising machine executable code which when executed by at least one machine, causes the at least one machine to:
write a first incremental change set in a write ahead log in a non-volatile memory,
wherein the first incremental change set indicates a change to metadata in volatile memory for a metadata client;

track a dependency of a checkpoint stored in the write ahead log to a second incremental change set,
wherein the second incremental change set corresponds to a region beyond a wrap point of the write ahead log; and

cause a new checkpoint for the metadata client to be persisted in the write ahead log, the new checkpoint having an updated dependency.

11. The non-transitory machine readable medium of claim 10 , further comprising code to cause the at least one machine to:
maintain the write ahead log as a circular buffer that wraps from an ending logical block address (LBA) to a starting LBA, the wrap point being between the ending LBA and the starting LBA and corresponding to a most recent dependency in the write ahead log.

12. The non-transitory machine readable medium of claim 10 , wherein code causing the at least one machine to track the dependency comprises:
code causing the at least one machine to determine that the checkpoint in the write ahead log records the dependency of the metadata client to an oldest incremental change set.

13. The non-transitory machine readable medium of claim 10 , wherein the code causing the at least one machine to cause the new checkpoint to be persisted includes code causing the at least one machine to persist the new checkpoint in response to an amount of free space in the write ahead log falling below a threshold.

14. The non-transitory machine readable medium of claim 10 , further comprising code causing the at least one machine to overwrite the second incremental change set subsequent to persisting the new checkpoint.

15. The non-transitory machine readable medium of claim 10 , further comprising code to cause the at least one machine to:
identify a point in the write ahead log as the wrap point where a first sequence number is associated with a first data extent and a second sequence number associated with a contiguous data extent are different from each other, wherein the first sequence number corresponds to a number of times that the wrap point has wrapped.

16. A storage controller comprising:
a memory containing machine readable medium comprising a non-transitory machine executable code having stored thereon instructions for performing a method of updating metadata in a storage system;
a processor coupled to the memory, the processor configured to execute the machine executable code to cause the processor to:
maintain a write ahead log comprising a plurality of incremental change sets in a non-volatile memory, a first incremental change set of the plurality of incremental change sets corresponding to a condition of metadata in volatile memory for a metadata client;
track a dependency of a checkpoint stored in the write ahead log to the first incremental change set, the dependency corresponding to a region beyond a wrap point of the write ahead log; and
cause a new checkpoint for the metadata client to be persisted in the write ahead log, the new checkpoint having an updated dependency.

17. The storage controller of claim 16 , the machine executable code further causing the processor to:
access a list maintained for the region of the write ahead log that identifies a plurality of metadata clients, including the metadata client, that maintain the checkpoint with the dependency on an oldest incremental change set.

18. The storage controller of claim 16 , the machine executable code further causing the processor to:
maintain a sequence number in the write ahead log, wherein the sequence number corresponds to a number of times that the wrap point has wrapped from an end of an address range to a start of the address range for the write ahead log.

19. The storage controller of claim 18 , the machine executable code further causing the processor to:
identify a point in the write ahead log as the wrap point where the sequence number is associated with a first data extent, and an additional sequence number associated with a contiguous data extent are different from each other.

20. The storage controller of claim 16 , wherein the machine executable code causing the new checkpoint to be persisted comprises machine executable code to cause the processor to persist the new checkpoint in response to an amount of free space in the write ahead log falling be

Assignees

• Netapp Inc

Inventors

• Sterns Randolph
• Binford Charles
• Delaney William
• Blount Joseph
• Moyer Keith
• Moore Joseph

Classifications

• G06F11/1471

  involving logging of persistent data for recovery · CPC title

• G06F2201/84

  Using snapshots, i.e. a logical point-in-time copy of the data · CPC title

• G06F17/40

  Data acquisition and logging (for input to computer G06F3/00) · CPC title

• G06F3/0619 Primary

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F3/0605

  by facilitating the interaction with a user or administrator · CPC title