Dynamic update to views of a file system backed by object storage

Patent metadata

Field	Value
Publication number	US-2016267103-A1
Application number	US-201615164465-A
Country	US
Kind code	A1
Filing date	May 25, 2016
Priority date	Mar 2, 2012
Publication date	Sep 15, 2016
Grant date	—

Abstract

A system can apply file placement rules to dynamically place files and directories within file system views backed by objects in an object storage system. After detection of an update to a first file system view that causes an update of an object in a storage grid, an object manager begins evaluation of file placement rules against metadata of the object. For each file placement rule that is triggered, the object manager determines identifies gateways that export the first file system view. The object manager then instructs the gateways to update their representations of the first file system view. The disclosed embodiments may be able to scale to managing hundreds of billions of files spanning thousands of file system views, especially in the presence of disconnected operation.

First claim

1 . A method comprising:
detecting an object storage operation for a first object in a distributed object storage system; after detection of the object storage operation, evaluating a plurality of file placement rules; based on the evaluation, identifying a first file placement rule for a first set of gateways that maintain local representations of a first namespace, wherein the first set of gateways is associated with the distributed object storage system; determining a first action to be performed based, at least in part, on the first namespace and the detected object storage operation for the first object; generating a notification that indicates an object identifier of the first object and that indicates the first action; and communicating the notification to the first set of gateways for the first set of gateways to update the local representations of the first namespace according to the notification.

2 . The method of claim 1 , further comprising identifying the first namespace from metadata of the first object.

3 . The method of claim 1 , wherein the first namespace is a sub-space of a file system namespace.

4 . The method of claim 1 , wherein the first object represents either a file system file or a file system directory.

5 . The method of claim 1 , further comprising:
detecting an update to a file system, wherein the object storage operation corresponds to the update to the file system and the first namespace is a sub-space of the file system's namespace; and based on detection of the update to the file system, notifying a second set of gateways about the update to cause the second set of gateways to update local representations of the first namespace based, at least in part, on the update, wherein the second set of gateways is associated with the distributed object storage system.

6 . The method of claim 1 , further comprising:
wherein the first action comprises a conflict resolution action to be performed by each of the first set of gateways that detects a conflict in the local representation thereof from the update.

7 . The method of claim 1 , wherein generating the notification comprises generating the notification with an indication of a hierarchical change in the first namespace based on the detected object storage operation.

8 . The method of claim 1 , wherein generating the notification comprises generating the notification also with an object identifier of a second object, wherein the second object represents a directory and the first object represents a file within the directory.

9 . The method of claim 1 , further comprising:
based on the evaluation, identifying a second file placement rule defined for the first namespace, wherein the second file placement rule indicates that an object created in the first namespace and matching a criterion is to be replicated to a second namespace; and communicating an instruction to a gateway associated with the second namespace, wherein the instruction causes the first object to be created in the second namespace.

10 . The method of claim 1 , further comprising:
determining a second action indicated by the first file placement rule; and determining that the object storage operations correspond to the first action and do not correspond to the second action.

11 . One or more non-transitory computer readable media comprising program instructions for synchronizing shares across a storage grid, the program instructions to:
after detection of an update to a first share that causes an update of a first object in the storage grid, begin evaluation of a plurality of file placement rules against metadata of the first object, wherein the first object corresponds to a file or a directory; based on the evaluation, determine that at least a first file placement rule is triggered by the update of the first object; identify a first set of gateways that export the first share to clients, wherein the first set of gateways is associated with the storage grid; and communicate an instruction to the first set of gateways to perform a first action defined in the first file placement rule.

12 . The non-transitory computer readable media of claim 11 , wherein the update to the first share comprises a create, delete, or modify of a file or a directory.

13 . The non-transitory computer readable media of claim 11 , further comprising program instructions to detect the update to the first share and to notify the first set of gateways of the update based on the first set of gateways previously subscribing for notification of updates to the first share.

14 . The non-transitory computer readable media of claim 11 , wherein the program instructions to evaluate comprise the program instructions to periodically evaluate the plurality of file placement rules.

15 . The non-transitory computer readable media of claim 11 , wherein the program instructions to evaluate comprise the program instructions to evaluate the plurality of file placement rules in response to detection of object creation or object destruction in the storage grid.

16 . The non-transitory computer readable media of claim 11 , wherein a plurality of shares corresponds to a file system namespace and the plurality of shares comprise the first share.

17 . The non-transitory computer readable media of claim 11 , wherein the program instructions to begin evaluation of the plurality of file placement rules against the first object comprise program instructions to determine whether metadata of the first object identifies a share specified by a criterion of the file placement rule.

18 . The non-transitory computer readable media of claim 11 , wherein the program instructions to communicate the instruction comprise the program instructions to communicate an instruction to perform a second action if a conflict will arise from the first action, wherein the second action is a conflict resolution action defined in the first file placement rule.

19 . An apparatus comprising:
a processor; a network interface; and a computer readable medium comprising program instructions executable by the processor to cause the apparatus to, after detection of an update to a first representation of a first shared view of a file system, begin evaluation of a plurality of file placement rules against metadata of a first object, wherein the update to the first representation causes an update in an object storage system for the first object; based on the evaluation, determine that at least a first file placement rule is triggered; identify a first set of gateways that maintain a respective one of a set of representations of the first shared view; and communicate, via the network interface, an instruction to the first set of gateways to perform a first action defined in the first file placement rule to synchronize the set of representations based, at least in part, on the update to the first representation.

20 . The apparatus of claim 19 , wherein the program instructions further comprise program instructions executable by the processor to cause the apparatus to:
detect a change to at least one of the plurality of file placement rules; in response to detection of a change to a file placement rule, evaluate the changed file placement rule against metadata of a plurality of objects in the object storage system that correspond to the file system; and communicate, via the network interface, instructions to gateways associated with the object storage system based, at least in part, on the evaluations.

Assignees

• Netapp Inc

Inventors

• Slik David
• Altman Tym
• Ciapponi Adam F

Classifications

• G06F16/183 Primary

  Provision of network file services by network file servers, e.g. by using NFS, CIFS (network file access protocols H04L67/1097) · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F16/185

  Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof (details of archiving G06F16/11) · CPC title

• G06F17/30203 Primary

  Physics · mapped topic