Systems and methods for reducing write tax, memory usage, and trapped capacity in metadata storage

Patent metadata

Field	Value
Publication number	US-11029862-B2
Application number	US-201715496206-A
Country	US
Kind code	B2
Filing date	Apr 25, 2017
Priority date	Apr 25, 2017
Publication date	Jun 8, 2021
Grant date	Jun 8, 2021

Abstract

A system and method for metadata storage includes use of a storage controller to receive a write operation from a host, create a metadata entry for storing metadata associated with data written by the write operation, and store the metadata entry into a metadata store. The metadata store includes a plurality of active partitions, a plurality of inactive partitions, and a queue identifying locations in the active partitions where the metadata entry may be stored. The metadata store is further used to access data written to one or more storage devices. Storing the metadata entry into the metadata store includes popping a first location from the queue, identifying a first active partition from the active partitions based on the first location, and storing the metadata entry in the first active partition. In some embodiments, each of the active partitions may be organized into two or more tiers.

First claim

What is claimed is:

1. A method comprising:
creating, by a storage controller, a metadata entry associated with data received with a write operation; and
storing, by the storage controller, the metadata entry into a metadata store, the metadata store comprising a plurality of active partitions and a plurality of inactive partitions including:
popping an index from a queue identifying locations in the plurality of active partitions where the metadata entry may be stored, each of the plurality of active partitions are allocated respective storage locations in a first tier storage pool and in a second tier storage pool, and each of the plurality of inactive partitions are allocated respective storage locations in the second tier storage pool and not allocated storage locations in the first tier storage pool;
identifying a first active partition from the plurality of active partitions and a first storage location within the first tier storage pool and associated with the first active partition based on the index; and
storing the metadata entry in the first storage location.

2. The method of claim 1 , further comprising:
flushing metadata entries stored in the first tier storage pool and associated with the first active partition, including merging the metadata entries into the second tier storage pool.

3. The method of claim 1 , further comprising:
flushing metadata entries stored in the first tier storage pool and associated with the first active partition in response to the first tier storage pool reaching a first size threshold.

4. The method of claim 1 , further comprising:
adding, in response to the queue no longer containing locations associated with the first active partition, locations associated with a second active partition of the plurality of active partitions to the queue.

5. The method of claim 1 , further comprising:
selecting, in response to the queue no longer containing locations associated with the first active partition, a first inactive partition from the plurality of inactive partitions to be a part of a second active partition.

6. The method of claim 1 , wherein the index comprises a first portion identifying the first active partition and a second portion identifying the first storage location.

7. The method of claim 1 , wherein the queue comprises a plurality of counters for tracking a number of indexes associated with each of the plurality of active partitions.

8. The method of claim 1 , wherein the queue comprises a plurality of second queues, each of the second queues identifying locations in respective ones of the plurality of active partitions.

9. A non-transitory machine-readable medium having stored thereon instructions for performing a method comprising machine executable code which when executed by at least one machine, causes the machine to:
create a metadata entry associated with data received with a write operation; and
store the metadata entry into a metadata store, the metadata store comprising a plurality of active partitions and a plurality of inactive partitions including:
pop an index from a queue identifying locations in the plurality of active partitions where the metadata entry may be stored, each of the plurality of active partitions are allocated respective storage locations in a first tier storage pool and in a second tier storage pool, and each of the plurality of inactive partitions are allocated respective storage locations in the second tier storage pool and not allocated storage locations in the first tier storage pool;
identify a first active partition from the plurality of active partitions and a first storage location within the first tier storage pool and associated with the first active partition based on the index; and
store the metadata entry in the first storage location.

10. The non-transitory machine-readable medium of claim 9 , wherein the machine is further caused to:
flush metadata entries stored in the first tier storage pool and associated with the first active partition, including merging the metadata entries into the second tier storage pool.

11. The non-transitory machine-readable medium of claim 9 , wherein the machine is further caused to:
flush metadata entries stored in the first tier storage pool and associated with the first active partition in response to the first tier storage pool.

12. The non-transitory machine-readable medium of claim 9 , wherein the machine is further caused to:
add, in response to the queue no longer containing locations associated with the first active partition, locations associated with a second active partition to the queue.

13. The non-transitory machine-readable medium of claim 9 , wherein the machine is further caused to:
select, in response to the queue no longer containing locations associated with the first active partition, a first inactive partition from the plurality of inactive partitions as a second active partition.

14. A computing device comprising:
a memory containing machine-readable medium comprising machine executable code having stored thereon instructions for performing a method of maintaining a metadata catalog; and
a processor coupled to the memory, the processor configured to execute the machine executable code to cause the processor to:
create a metadata object for storing metadata associated with data received with a write operation; and
store the metadata object into the metadata catalog, the metadata catalog comprising a plurality of active portions and a plurality of inactive portions including:
extract a first index from a list identifying locations in the plurality of active portions where the metadata object may be stored, each of the plurality of active portions are allocated respective storage locations in a first layer storage pool and a second layer storage pool, and each of the plurality of inactive portions are allocated respective storage locations in the second layer storage pool and not allocated storage locations in the first layer storage pool;
identify a first active portion from the plurality of active portions and a first storage location within the first layer storage pool and associated with the first active portion based on the first index; and
store the metadata object in the first storage location.

15. The computing device of claim 14 , wherein the processor is further caused to:
flush metadata entries stored in the first layer storage pool and associated with the first active portion, including merging the metadata entries into the second layer storage pool.

16. The computing device of claim 14 , wherein the processor is further caused to:
add, in response to the list no longer containing indexes associated with the first active portion, indexes associated with a second active portion of the plurality of active portions to the list.

17. The computing device of claim 14 , wherein the processor is further caused to:
select, in response to the list no longer containing indexes associated with the first active portion, a first inactive portion from the plurality of inactive portions to be a part of a second active portion.

18. The method of claim 1 , further comprising:
reassigning a portion of the first tier storage pool that is associated with the first active partition to a second active partition in response to the queue no longer containing locations associated with the first active partition.

19. The computing device of claim 14 , wherein the processor is further caused to:
determine, based on a size of a portion of the second layer stora

Assignees

• Netapp Inc

Inventors

• Blount Joseph

Classifications

• G06F3/0689

  Disk arrays, e.g. RAID, JBOD · CPC title

• G06F3/0631 Primary

  by allocating resources to storage systems · CPC title

• G06F3/061

  Improving I/O performance · CPC title