File system for efficient object fragment access

Patent metadata

Field	Value
Publication number	US-9767104-B2
Application number	US-201414475332-A
Country	US
Kind code	B2
Filing date	Sep 2, 2014
Priority date	Sep 2, 2014
Publication date	Sep 19, 2017
Grant date	Sep 19, 2017

Abstract

Technology is disclosed for accessing data fragments of data objects. The method receives a request for storing a data fragment of a data object in the storage server. The request includes an object identifier of the data object. The method further extracts a first string from the object identifier. The method then determines whether there is an existing file system object having a file system name that matches the first string. If there is no file system object that has a file system name that matches the first string, the method stores the data fragment as a fragment file with a file system name matching the first string.

First claim

What is claimed is:

1. A method, comprising:
based on receipt of a request to store a data fragment of a data object at a storage node of a distributed storage system, determining whether a file system of the storage node has an existing file system object having a file system name that matches a first section of an object identifier of the data object;
storing, in the file system of the storage node, the data fragment as a file with a file system name that matches the first section of the object identifier, based on a determination that there is no file system object in the file system that matches the first section; and
based on a determination that the file system includes a file system object with a file system name that matches the first section of the object identifier,
creating a directory with the file system name that matches the first section; and
storing within the directory the data fragment as a file having a file name including a second section of the object identifier.

2. The method of claim 1 , further comprising:
determining, by the storage node, that the file system object having the file system name that matches the first section of the object identifier is a file;
renaming the file having the file system name that matches the first section to include the second section of the object identifier; and
moving the renamed file into the directory.

3. The method of claim 2 ,
wherein at least one of renaming the file and storing the data fragment comprises differentiating the file names of the renamed file and the file.

4. The method of claim 1 , further comprising:
returning, by the storage node, a response confirming that the data fragment of the data object has been successfully stored in the storage node.

5. The method of claim 1 , wherein the determining whether a file system of the storage node has an existing file system object having a file system name that matches a first section of an object identifier of the data object comprises:
linearly searching a file system name index.

6. The method of claim 1 , wherein storing the data fragment as the file comprises:
inserting, by the storage node, fragment metadata into the file.

7. The method of claim 6 , wherein the fragment metadata comprises:
the object identifier of the data object;
a version number of the data object;
a segment identifier of a data segment to which the data fragment belongs; and
a fragment identifier indicating a relative position of the data fragment within the data segment.

8. A non-transitory machine readable medium comprising instructions for maintaining object fragments in a file system of a storage node of a distributed storage system, the instructions executable to:
based on receipt of a request to read a data fragment of a data object from a storage node, determine whether the file system of the storage node has an existing file system object having a file system name that matches a first section of an object identifier for the data object;
based on a determination that the file system has an existing file system object having the file system name that matches the first section, determine whether the file system object is a file or a directory;
based on a determination that the file system object having the file system name that matches the first section is a file, read fragment metadata from the file to verify that the fragment metadata comprises the object identifier;
based on a determination that the fragment metadata read from the file comprises the object identifier, return the file in response to the request; and
based on a determination that the file system object having the file system name that matches the first section is a directory, determine which files within the directory comprises fragment metadata that comprise the object identifier and return the ones of the files determined to comprise fragment metadata that comprise the object identifier.

9. The non-transitory machine readable medium of claim 8 further comprising instructions executable to:
based on a determination that the file system does not have a file system object having the file system name that matches the first section, return a response message indicating that no data fragment of the data object is available from the storage node.

10. The non-transitory machine readable medium of claim 8 to further comprising instructions executable to:
based on a determination that the file system object having the file system name that matches the first section is a directory, identify files within the directory having a file system name that includes a second section of the object identifier,
wherein the instructions executable to determine which files within the directory comprises fragment metadata that comprise the object identifier comprise the instructions executable to read the fragment metadata from the files identified as having a file system name that includes the second section.

11. The non-transitory machine readable medium of claim 8 , wherein the instructions executable to determine whether the file system of the storage node has an existing file system object having a file system name that matches a first section of an object identifier for the data object comprise instructions executable to:
conduct a linear search in a file system name index for the first section.

12. A computing device, comprising:
a processor; and
a machine readable medium comprising program code executable by the processor to cause the computing device to:
based on receipt of a request to access data of a data object identified by an object identifier that comprises at least a first section and a second section, determine whether a file system of the computing device includes a file system object with a file system name that includes the first section of the object identifier;
based on a determination that the file system includes a file system object having the file system name that includes the first section and that the file system object is a file, return the file as part of the data object; and
based on a determination that the file system includes a file system object having the file system name that includes the first section and that the file system object is a directory, verify that each of a plurality of files within the directory comprises fragment metadata that comprise the object identifier and return verified ones of the plurality of files as parts of the data object.

13. The computing device of claim 12 , wherein the machine readable medium further comprises program code executable by the processor to cause the computing device to:
verify that the file having the file system name that includes the first section comprises fragment metadata that comprise the object identifier;
wherein return of the file as part of the data object is also based on verification of the file.

14. The computing device of claim 12 , wherein machine readable medium further comprises code executable by the processor to cause the computing device to:
determine the plurality of files as files within the directory having a file name that includes the second section.

15. The computing device of claim 12 , wherein the data object includes multiple segments, at least one of the segments includes multiple fragments, wherein the fragments are stored in a distributed storage system that includes the computing device and at least one of the fragments is stored in the computing device.

16. The computing device of claim 12 , wherein the data object is erasure coded for storing into a distributed storage system that includes

Assignees

• Netapp Inc

Inventors

• Benight Barry Patrick

Classifications

• G06F16/122 Primary

  using management policies (point-in-time backing up or restoration of persistent data G06F11/1446; file migration policies for HSM systems G06F16/185) · CPC title

• G06F16/164

  File meta data generation · CPC title

• G06F17/30082 Primary

  Physics · mapped topic

• G06F17/3012

  Physics · mapped topic