Object storage backed file system

Patent metadata

Field	Value
Publication number	US-10929419-B2
Application number	US-201514866793-A
Country	US
Kind code	B2
Filing date	Sep 25, 2015
Priority date	Sep 25, 2015
Publication date	Feb 23, 2021
Grant date	Feb 23, 2021

Abstract

To leverage the attributes of object storage for applications/systems created to interface with a network files system, an object storage backed file system can accept the defined file system commands from the applications/systems and transform the file system commands into requests that target object storage. The file system is “backed” by object storage because attributes and content of file system entities are stored in objects. For instance, content data and metadata of a file are stored in objects in object storage. This object storage backed file system can be considered a bridge between a client perceived hierarchical file system namespace and a flat namespace of an object storage.

First claim

What is claimed is:

1. A method comprising:
generating, by an object storage-backed file system (OSFS) in response to receiving a file system command that specifies creation of a file system entity from a client, a first object storage request to create a first object in an object storage with a first object key derived from an identifying number of the file system entity and with metadata that indicates attributes of the file system entity;
generating, by the OSFS in response to receiving the file system command, a second object storage request to create a second object in the object storage with a second object key and with metadata that associates the second object key with the first object key, wherein the second object key is based on an identifying number of a parent directory of the file system entity and a name of the file system entity;
receiving, by the OSFS from an object store, indications that the first and second object storage requests have successfully completed in the object storage; and
providing, by the OSFS in response to receiving the indication, a response to the client that the file system command has completed.

2. The method of claim 1 further comprising:
associating the first object storage request and the second object storage request with a transaction identifier.

3. The method of claim 1 wherein the metadata that associates the second object key with the first object key comprises metadata that maps the second object key to the first object key.

4. The method of claim 1 further comprising:
converting the identifying number of the file system entity from a number into a text representation of the identifying number of the file system entity to be the first object key.

5. The method of claim 1 wherein the file system entity is a file.

6. The method of claim 1 further comprising:
communicating, by the OSFS, a transaction corresponding to the first object storage request and the second object storage request to a file system caching mechanism.

7. The method of claim 1 further comprising:
preventing, by the OSFS, eviction of data of the first and second objects from a cache of the OSFS until the first and second objects are replicated across a threshold number of nodes of the object storage.

8. The method of claim 1 further comprising:
allowing, by the OSFS, data of the first and second objects to be partially evicted from a cache of the OSFS after the first and second objects have been replicated across a threshold number of nodes of the object storage.

9. One or more non-transitory machine-readable storage media having program code for an object storage-backed file system (OSFS), the program code comprising instructions to:
create, by the OSFS, in object storage a first object with a first object key derived from an identifying number of an entity, in a hierarchal file system namespace, with metadata that indicates attributes of the entity;
create, by the OSFS, in object storage a second object with a second object key and with metadata that associates the second object key with the first object key, wherein the second object key is based on an identifying number of a parent directory of the entity and a name of the entity;
transform, by the OSFS, a file system command relating to the entity into one or more object storage requests corresponding to the first object and the second object; and
generate, by the OSFS, a response to the file system command based, at least in part, on an object storage response to the one or more object storage requests.

10. The machine-readable storage media of claim 9 , further comprising instructions to:
submit, by the OSFS, a transaction corresponding to the one or more object storage requests to a caching mechanism of the OSFS.

11. The machine-readable storage media of claim 9 , wherein the instructions to transform the file system command comprise instructions to:
generate, by the OSFS from among the one or more object storage requests, an object storage request that retrieves metadata of the first object for the entity specified in the file system command to validate the file system command.

12. The machine-readable storage media of claim 9 , wherein the instructions to transform the file system command comprise instructions to:
generate, for the file system command comprising a move of the entity from the parent directory to a destination directory, an object storage request that creates a third object with a third object key and with metadata that associates the third object key with the first object key, wherein the third object key is based on the identifying number of the destination directory and the name of the entity.

13. The machine-readable storage media of claim 9 , wherein the instructions further comprise instructions to:
generate an object storage request that retrieves metadata of the first object for the entity to validate the file system command.

14. The machine-readable storage media of claim 9 , wherein the instructions further comprise instructions to:
verify that the metadata of the first object for a destination directory indicates that the first object represents a directory.

15. The machine-readable storage media of claim 9 , wherein the instructions to transform the file system command comprise instructions to:
generate, for the file system command comprising a rename of the entity from a first name to a second name, an object storage request that creates a third object with a third object key and with metadata that associates the third object key with the first object key, wherein the third object key is based on the identifying number of the parent directory and the second name.

16. An apparatus comprising:
a processor unit; and
a machine-readable medium having program code executable by the processor unit to cause the apparatus to:
create, in object storage comprising a flat namespace, a first object with a first object key derived from an identifying number of an entity in a hierarchal file system namespace, with metadata that indicates attributes of the entity;
create, in the object storage, a second object with a second object key and with metadata that associates the second object key with the first object key, wherein the second object key is based on an identifying number of a parent directory of the entity and a name of the entity;
transform a file system command relating to the entity and according to the hierarchal file system namespace into one or more object storage requests according to the flat namespace and corresponding to the first object and the second object; and
generate responses to the file system command based, at least in part, on an object storage response to the one or more object storage requests.

17. The apparatus of claim 16 , wherein the program code is further executable by the processor unit to cause the apparatus to:
submit a transaction corresponding to the one or more object storage requests to a caching mechanism of the object storage backed file system.

18. The apparatus of claim 16 , wherein the program code to transform the file system command comprises program code executable by the processor unit to cause the apparatus to:
generate, from among the one or more object storage requests, an object storage request that retrieves metadata of the first object for the entity specified in the file system command to validate the file system command.

19. The apparatus of claim 16 , wherein the program code to transform the file system command comprises program code executabl

Assignees

• Netapp Inc

Inventors

• Beard Derek William
• Sirton Guy
• Hayes Garret Lance
• Preston Jason Dillard

Classifications

• G06F16/182 Primary

  Distributed file systems · CPC title

• G06F16/183

  Provision of network file services by network file servers, e.g. by using NFS, CIFS (network file access protocols H04L67/1097) · CPC title

• G06F16/1865

  Transactional file systems · CPC title

• G06F16/258 Primary

  Data format conversion from or to a database · CPC title

• G06F16/2228

  Indexing structures · CPC title