Dynamic update to views of a file system backed by object storage

Patent metadata

Field	Value
Publication number	US-11853265-B2
Application number	US-202016943422-A
Country	US
Kind code	B2
Filing date	Jul 30, 2020
Priority date	Mar 2, 2012
Publication date	Dec 26, 2023
Grant date	Dec 26, 2023

Abstract

A system can apply file placement rules to dynamically place files and directories within file system views backed by objects in an object storage system. After detection of an update to a first file system view that causes an update of an object in a storage grid, an object manager begins evaluation of file placement rules against metadata of the object. For each file placement rule that is triggered, the object manager determines identifies gateways that export the first file system view. The object manager then instructs the gateways to update their representations of the first file system view. The disclosed embodiments may be able to scale to managing hundreds of billions of files spanning thousands of file system views, especially in the presence of disconnected operation.

First claim

The invention claimed is:

1. A method, comprising:
receiving, by a distributed object storage system, a storage manipulation operation for a file in response to a command from a client system;
updating, by the distributed object storage system, an object in an object store, the object associated with the file and the updating the object being based on the storage manipulation operation for the file;
identifying, by the distributed object storage system, a file placement rule as corresponding to the file, the file placement rule specifying how to place the file across one or more addressable namespaces corresponding to one or more file system views;
generating, by the distributed object storage system, a notification comprising information about the object corresponding to the file, the information including an object identifier and file metadata to allow one or more gateways to update a file system view of the one or more file system views at the one or more gateways; and
transmitting, by the distributed object storage system based on the file placement rule, the notification to the one or more gateways associated with the distributed object storage system to keep the one or more file system views at the corresponding one or more gateways up to date.

2. The method of claim 1 , further comprising:
instructing, by the distributed object storage system, the one or more gateways with information about the object to invalidate a cache at the one or more gateways.

3. The method of claim 1 , further comprising:
queuing, by the distributed object storage system, the notification for transmission in response to the one or more gateways becoming unavailable until the one or more gateways is available again.

4. The method of claim 1 , wherein the file placement rule comprises a first file placement rule, the method further comprising:
identifying, by the distributed object storage system, a second file placement rule as corresponding to the file.

5. The method of claim 1 , further comprising:
resending, by the distributed object storage system, the notification to the one or more gateways on a periodic basis.

6. The method of claim 1 , wherein the distributed object storage system comprises an object manager which performs the receiving, updating, identifying, generating, and transmitting.

7. A non-transitory machine readable medium having stored thereon instructions for performing a method comprising machine executable code which when executed by at least one machine, causes the machine to:
receive a storage manipulation operation from a client system for a file in an addressable namespace;
update an object in an object store based on the storage manipulation operation, the object associated with the file;
identify a file placement rule from among a plurality of file placement rules as corresponding to the file based on a criterion of the file placement rule matching the file, the file placement rule specifying how to place the file across one or more addressable namespaces corresponding to one or more file system views;
generate a notification comprising an object identifier and file metadata as information about the object to allow the one or more gateways to update a file system view of the one or more file system views at the one or more gateways; and
transmit, based on the file placement rule, the notification to the one or more gateways associated with the at least one machine within a distributed object storage system to keep the one or more file system views at the corresponding one or more gateways up to date.

8. The non-transitory machine readable medium of claim 7 , further comprising machine executable code which when executed by the at least one machine, causes the machine to:
instruct the one or more gateways with information about the object to invalidate a cache at the one or more gateways.

9. The non-transitory machine readable medium of claim 7 , further comprising machine executable code which when executed by the at least one machine, causes the machine to:
queue the notification for transmission in response to the one or more gateways becoming unavailable until the one or more gateways is available again.

10. The non-transitory machine readable medium of claim 7 , further comprising machine executable code which when executed by the at least one machine, causes the machine to:
resend the notification to the one or more gateways on a periodic basis.

11. The non-transitory machine readable medium of claim 7 , wherein the machine comprises an object manager.

12. A computing device, comprising:
a network interface;
a memory containing machine readable medium comprising machine executable code having stored thereon instructions for performing a method of synchronization across one or more gateways associated with a distributed object storage system; and
a processor coupled to the memory and the network interface, the processor configured to execute the machine executable code to cause the processor to:
receive a storage manipulation operation for a file in response to a command from a client system;
update an object in an object store based on the storage manipulation operation for the file, and the object associated with the file;
identify a file placement rule as corresponding to the file, the file placement rule specifying how to place the file across one or more addressable namespaces corresponding to one or more file system views;
generate a notification comprising information about the object corresponding to the file, the information including an object identifier and file metadata to allow one or more gateways to update a file system view of the one or more file system views at the one or more gateways; and
transmit, based on the file placement rule, the notification to the one or more gateways associated with the distributed object storage system to keep the one or more file system views at the corresponding one or more gateways up to date.

13. The computing device of claim 12 , wherein execution of the machine executable code further causes the processor to:
instruct the one or more gateways with information about the object to invalidate a cache at the one or more gateways.

14. The computing device of claim 12 , wherein execution of the machine executable code further causes the processor to:
queue the notification for transmission in response to the one or more gateways becoming unavailable until the one or more gateways is available again.

15. The computing device of claim 12 , wherein:
the file placement rule comprises a first file placement rule, and
execution of the machine executable code further causes the processor to identify a second file placement rule as corresponding to the file.

16. The computing device of claim 12 , wherein execution of the machine executable code further causes the processor to:
resend the notification to the one or more gateways on a periodic basis.

17. The computing device of claim 12 , wherein the computing device comprises an object manager.

Assignees

• Netapp Inc

Inventors

• Slik David
• Altman Tym
• Ciapponi Adam F

Classifications

• G06F16/183 Primary

  Provision of network file services by network file servers, e.g. by using NFS, CIFS (network file access protocols H04L67/1097) · CPC title

• G06F16/185

  Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof (details of archiving G06F16/11) · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title