Managing changes to metadata in a data storage system

Patent metadata

Field	Value
Publication number	US-12430210-B2
Application number	US-202318188568-A
Country	US
Kind code	B2
Filing date	Mar 23, 2023
Priority date	Mar 23, 2023
Publication date	Sep 30, 2025
Grant date	Sep 30, 2025

Abstract

A searchable metadata change log is maintained in a persistent memory of a data storage system, and stores metadata changes to be aggregated and applied to data storage system metadata located in non-volatile data storage. New metadata changes are added to the metadata change log by storing the new metadata changes into the persistent memory, storing structural modifications to the metadata change log needed to add the new metadata changes to the metadata change log into a transaction entry located in the persistent memory, and processing the resulting transaction entry to perform the structural modifications to the metadata change log that were stored in the transaction entry.

First claim

What is claimed is:

1. A method comprising:
maintaining, in a persistent memory of a data storage system, a metadata change log, wherein the metadata change log is searchable and stores metadata changes that are to be applied to data storage system metadata that is stored in non-volatile data storage devices of the data storage system; and
adding a plurality of new metadata changes to the metadata change log, by:
storing the new metadata changes into the persistent memory within a memory pool for the metadata change log, wherein the memory pool for the metadata change log comprises a portion of the persistent memory from which the metadata change log is allocated,
storing, into a transaction entry located in the persistent memory within a memory pool for transaction entries, a plurality of structural modifications to the metadata change log that are needed to add the new metadata changes to the metadata change log, by storing, for each one of the structural modifications, i) an offset of a null pointer within the memory pool for the metadata change log and ii) a pointer to a location of a respective one of the new metadata changes within the persistent memory, wherein the null pointer is replaced by the pointer to the location of the respective one of the new metadata changes when the transaction entry is processed, and wherein the memory pool for transaction entries comprises a portion of the persistent memory from which transaction entries are allocated that is separately located from the portion of the persistent memory from which the metadata change log is allocated, and
processing the transaction entry to perform the structural modifications to the metadata change log stored in the transaction entry.

2. The method of claim 1 , wherein the transaction entry is processed atomically, such that successful processing of the transaction entry is indicated only responsive to all structural modifications to the metadata change log that are stored in the transaction entry having been performed.

3. The method of claim 2 , wherein the transaction entry is one of a plurality of transaction entries stored in the persistent memory; and
wherein recovering from a failure event during which contents of volatile memory of the data storage system was lost includes, for each one of the plurality of transaction entries:
determining whether the transaction entry is valid;
in response to determining that the transaction entry is valid, determining whether processing of the transaction entry was interrupted by the failure event; and
in response to determining that processing of the transaction entry was interrupted by the failure event, completing processing of the transaction entry.

4. The method of claim 1 , wherein the persistent memory is battery backed to prevent loss of contents of the persistent memory during failure events that cause contents of a volatile memory of the data storage system to be lost.

5. The method of claim 1 , wherein the persistent memory is located in a first storage processor of the data storage system; and
wherein contents of the persistent memory is mirrored to another persistent memory located in a second storage processor of the data storage system.

6. The method of claim 1 , wherein the non-volatile data storage devices of the data storage system comprise at least one solid state drive.

7. A data storage system comprising:
processing circuitry;
a memory;
a persistent memory;
a plurality of non-volatile data storage drives; and
wherein the memory has program code stored thereon, wherein the program code, when executed by the processing circuitry, causes the processing circuitry to:
maintain, in the persistent memory, a metadata change log, wherein the metadata change log is searchable and stores metadata changes that are to be applied to data storage system metadata that is stored in non-volatile data storage devices of the data storage system; and
add a plurality of new metadata changes to the metadata change log, by causing the processing circuitry to:
store the new metadata changes into the persistent memory within a memory pool for the metadata change log, wherein the memory pool for the metadata change log comprises a portion of the persistent memory from which the metadata change log is allocated,
store, into a transaction entry located in the persistent memory within a memory pool for transaction entries, a plurality of structural modifications to the metadata change log that are needed to add the new metadata changes to the metadata change log, by causing the processing circuitry to store, for each one of the structural modifications, i) an offset of a null pointer within the memory pool for the metadata change log and ii) a pointer to a location of a respective one of the new metadata changes within the persistent memory, wherein the null pointer is replaced by the pointer to the location of the respective one of the new metadata changes when the transaction entry is processed, and wherein the memory pool for transaction entries comprises a portion of the persistent memory from which transaction entries are allocated that is separately located from the portion of the persistent memory from which the metadata change log is allocated, and
process the transaction entry to perform the structural modifications to the metadata change log stored in the transaction entry.

8. The data storage system of claim 7 , wherein the transaction entry is processed atomically, such that successful processing of the transaction entry is indicated only responsive to all structural modifications to the metadata change log that are stored in the transaction entry having been performed.

9. The data storage system of claim 8 , wherein the transaction entry is one of a plurality of transaction entries stored in the persistent memory; and
wherein the program code further comprises program code that, when executed by the processing circuitry, causes the processing circuitry to recover from a failure event during which contents of volatile memory of the data storage system was lost at least in part by causing the processing circuitry to, for each one of the plurality of transaction entries:
determine whether the transaction entry is valid;
in response to a determination that the transaction entry is valid, determine whether processing of the transaction entry was interrupted by the failure event; and
in response to a determination that processing of the transaction entry was interrupted by the failure event, complete processing of the transaction entry.

10. The data storage system of claim 7 , wherein the persistent memory is battery backed to prevent loss of contents of the persistent memory during failure events that cause contents of the memory to be lost.

11. The data storage system of claim 7 , wherein the persistent memory is located in a first storage processor of the data storage system; and
wherein contents of the persistent memory is mirrored to another persistent memory located in a second storage processor of the data storage system.

12. The data storage system of claim 7 , wherein the non-volatile data storage devices of the data storage system comprise at least one solid state drive.

13. A computer program product including a non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed on processing circuitry, cause the processing circuitry to perform steps including:
maintaining, in a persistent memory of a data storage system, a metadata change log, wherein the metadata change log is searchable and stores metadata changes that are to be applied to data storage syste

Assignees

• Dell Products Lp

Inventors

• Zalstein Dror
• Shveidel Vladimir
• Derzhavetz Jenny

Classifications

• G06F11/0793

  Remedial or corrective actions (recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40) · CPC title

• G06F11/2056

  by mirroring · CPC title

• G06F11/1435 Primary

  using file system or storage system metadata · CPC title