Cache-based file system on object storage

Patent metadata

Field	Value
Publication number	US-12517684-B2
Application number	US-202418644902-A
Country	US
Kind code	B2
Filing date	Apr 24, 2024
Priority date	Apr 24, 2024
Publication date	Jan 6, 2026
Grant date	Jan 6, 2026

Abstract

Systems and methods for using object storage as a primary storage tier for a file system of a virtual storage system are provided. According to one embodiment, ephemeral storage of a host on which a virtual storage system is running in the cloud is used as an external cache in which all or some subset of file system metadata may be stored, for example, to boost read IOPS and reduce latency of the read path. In various examples, external cache policies may be defined to trigger adding to, updating, and/or re-warming the external cache to accelerate access to file system metadata, thereby making a virtual storage solution that maintains all file system data and file system metadata within object storage a suitable and cost-effective platform for certain types of workloads. In some example, the working set of a workload may also be kept in the external cache to further accelerate read performance of the virtual storage system.

First claim

What is claimed is:

1 . A virtual storage system comprising:
one or more processing resource; and instructions that when executed by the one or more processing resources cause the virtual storage system to: utilize an object storage service provided by a hyperscaler within which the virtual storage system is deployed as a primary storage tier; and accelerate read performance of a file system of the virtual storage system by maintaining, within ephemeral storage associated with a host on which the virtual storage system is deployed, a cache containing metadata of the file system associated with data previously read by a client of the virtual storage system, wherein said maintaining includes:
inserting a first new block of metadata into the cache after the first new block of metadata has been created by the file system;
inserting a second new block of metadata into the cache after the second new block of metadata is read from object storage as part of a read of client data; and
updating an existing block of metadata after the existing block of metadata has been overwritten by the file system.

2 . The virtual storage system of claim 1 , wherein the file system comprises a block-based file system.

3 . The virtual storage system of claim 2 , wherein all writes performed by the file system are performed to free blocks in the file system.

4 . The virtual storage system of claim 2 , wherein the instructions further cause the virtual storage system to map block numbers of an aggregate of the file system to objects stored within the object storage service.

5 . The virtual storage system of claim 1 , wherein the object storage service implements a Simple Storage Service (S3) protocol.

6 . The virtual storage system of claim 1 , wherein the instructions further cause the virtual storage system to, prior to an imminent shutdown or a reboot of the virtual storage system, facilitate re-warming of the cache by flushing pointers to cached metadata objects to special objects persisted within the object storage service.

7 . The virtual storage system of claim 6 , wherein the instructions further cause the virtual storage system to, after a restart of the virtual storage system, re-warm the cache by reloading the cached metadata objects.

8 . The virtual storage system of claim 7 , wherein re-warming of the cache is performed after a determination has been made that the virtual storage system has been rehosted.

9 . The virtual storage system of claim 6 , wherein the instructions further cause the virtual storage system to, after a restart of the virtual storage system:
determine the virtual storage system has been brought up on the host of the hyperscaler on which the virtual storage system was running prior to the restart; and based on said determining, continuing to make use of the cached file system metadata already residing within the cache.

10 . The virtual storage system of claim 1 , wherein the instructions further cause the virtual storage system to maintain within the cache a working set of a workload of the client.

11 . The virtual storage system of claim 1 , wherein the file system comprises a write-anywhere file system.

12 . The virtual storage system of claim 1 , wherein the metadata includes one or more of information regarding one or more container files in which the client data is stored, information indicating which physical volume block numbers (PVBNs) of an aggregate of the file system are in use, and information regarding a hierarchical set of index node (inode) data structures representing a state of the file system.

13 . A method comprising:
utilizing, by a virtual storage system deployed within a hyperscaler, an object storage service provided by the hyperscaler as a primary storage tier; and accelerating read performance of a file system of the virtual storage system by maintaining, within ephemeral storage associated with a host on which the virtual storage system is deployed, a cache containing metadata of the file system associated with data previously read by a client of the virtual storage system, wherein said maintaining includes:
inserting a first new block of metadata into the cache after the first new block of metadata has been created by the file system;
inserting a second new block of metadata into the cache after the second new block of metadata is read from object storage as part of a read of client data; and
updating an existing block of metadata after the existing block of metadata has been overwritten by the file system.

14 . The method of claim 13 , wherein the file system comprises a block-based file system and wherein all writes performed by the file system are performed to free blocks in the file system.

15 . The method of claim 13 , wherein the file system comprises a block-based file system and wherein the method further comprises mapping block numbers of an aggregate of the file system to objects stored within the object storage service.

16 . The method of claim 13 , further comprising:
prior to an imminent shutdown or a reboot of the virtual storage system, facilitating re-warming of the cache by flushing pointers to cached metadata objects to special objects persisted within the object storage service; and after a restart of the virtual storage system, re-warming the cache by reloading the cached metadata objects.

17 . The method of claim 13 , further comprising maintaining within the cache a working set of a workload of the client.

18 . The method of claim 13 , wherein the file system comprises a write-anywhere file system.

19 . The method of claim 13 , wherein the metadata includes one or more of information regarding one or more container files in which the client data is stored, information indicating which physical volume block numbers (PVBNs) of an aggregate of the file system are in use, and information regarding a hierarchical set of index node (inode) data structures representing a state of the file system.

20 . A non-transitory machine readable medium storing instructions, which when executed by one or more processing resources of a virtual storage system deployed within a hyperscaler, cause the virtual storage system to:
utilize, by the virtual storage system, an object storage service provided by the hyperscaler as a primary storage tier; and accelerate read performance of a file system of the virtual storage system by maintaining, within ephemeral storage associated with a host on which the virtual storage system is deployed, a cache containing metadata of the file system associated with data previously read by a client of the virtual storage system, wherein said maintaining includes:
inserting a first new block of metadata into the cache after the first new block of metadata has been created by the file system;
inserting a second new block of metadata into the cache after the second new block of metadata is read from object storage as part of a read of client data; and
updating an existing block of metadata after the existing block of metadata has been overwritten by the file system.

21 . The non-transitory machine readable medium of claim 20 , wherein the file system comprises a block-based file system and wherein all writes performed by the file system are performed to free blocks in the file system.

22 . The non-transitory machine readable medium of claim 20 , wherein the file system comprises a block-based file system and wherein the instructions further cause the vi

Assignees

• Netapp Inc

Inventors

• Bhattacharjee Mrinal K
• Banerjee Arindam
• Kharai Arjun Singh
• Pawar Dnyaneshwar

Classifications

• G06F3/0604

  Improving or facilitating administration, e.g. storage management · CPC title

• G06F3/064 Primary

  Management of blocks · CPC title

• G06F3/0673

  Single storage device · CPC title

• G06F3/0667 Primary

  at data level, e.g. file, record or object virtualisation · CPC title