Intelligent file system with transparent storage tiering

Patent metadata

Field	Value
Publication number	US-11609884-B2
Application number	US-202117187480-A
Country	US
Kind code	B2
Filing date	Feb 26, 2021
Priority date	Dec 15, 2014
Publication date	Mar 21, 2023
Grant date	Mar 21, 2023

Abstract

A file system manager implemented at a provider network identifies a storage device of a first group of storage devices of a provider network as an initial location of a file system object. Based on an access metric associated with the object, the file system manager initiates a transfer of contents of the object to a second storage device of a different storage device group, without receiving a client request specifying the transfer. In response to an access request received via a file system programmatic interface, contents of the object are provided from the second storage device. Based on a second access metric, the object is transferred back to the first group of storage devices.

First claim

What is claimed is:

1. A system, comprising:
a first tier of object storage devices with a first set of characteristics with a first set of values;
a second tier of object storage devices with a second set of characteristics with a second set of values, wherein the first set of values is different in at least one respect from the second set of values; and
one or more computing devices comprising memory storing executable instructions that, as a result of execution by one or more processors, cause the one or more computing devices to:
receive a first file system object via a network;
store the first file system object in a first object storage device of the first tier of object storage devices;
collect one or more metrics associated with the first file system object, including a first metric;
detect, based at least upon the first metric, that a condition has been met;
initiate, based at least in part on the detection that the condition has been met, a first transfer of at least a portion of contents of the first file system object from the first object storage device to a second object storage device of the second tier of object storage devices, independent of a client-initiated transfer request specific to the first file system object from a client of a file system to which the first file system object belongs, to make a first portion of the first object storage device available for use, wherein a difference between the first set of values and the second set of values provides a benefit of transferring the at least a portion of contents of the first file system object from the first tier to the second tier;
determine, based on the first set of characteristics, that the first tier of object storage devices should be used to store a second file system object; and
store the second file system object in the first portion of the first object storage device.

2. The system as recited in claim 1 , wherein the first set of characteristics includes at least a first speed to access data from the first tier of object storage devices, and wherein the second set of characteristics includes at least a second speed to access data from the second tier of object storage devices.

3. The system as recited in claim 2 , wherein the first speed is greater than the second speed.

4. The system as recited in claim 1 , wherein the benefit is that a cost of storing the first file system object on the second tier of object storage devices is less that a cost of storing the first file system object on the first tier of object storage devices.

5. The system as recited in claim 1 , wherein a second metric indicates that the second file system object is likely to be accessed more frequently than the first file system object.

6. The system as recited in claim 1 , wherein the first set of characteristics includes at least a first cost, and wherein the second set of characteristics includes at least a second cost.

7. The system as recited in claim 6 , wherein the second cost is less than the first cost.

8. A computer-implemented method, comprising:
receiving a first file system object via a network;
storing the first file system object in a first object storage device of a first tier of object storage devices of a plurality of tiers of object storage devices, wherein the plurality of tiers includes a second tier of object storage devices, wherein the first tier has a first set of characteristics with a first set of values, wherein the second tier has a second set of characteristics with a second set of values, and wherein the first set of values is different in at least one respect from the second set of values;
collecting one or more metrics associated with the first file system object, including a first metric;
detecting, based at least upon the first metric, that a condition has been met;
initiating, based at least in part on detecting that the condition has been met, a first transfer of at least a portion of contents of the first file system object from the first object storage device to a second object storage device of the second tier of object storage devices, independent of a client-initiated transfer request specific to the first file system object from a client of a file system to which the first file system object belongs, to make a first portion of the first object storage device available for use, wherein a difference between the first set of values and the second set of values provides a benefit of transferring the at least a portion of contents of the first file system object from the first tier to the second tier;
determining, based on the first set of characteristics, that the first tier of object storage devices should be used to store a second file system object; and
storing the second file system object in the first portion of the first object storage device.

9. The computer-implemented method as recited in claim 8 , wherein the first set of characteristics includes at least a first speed to access data from the first tier of object storage devices, and wherein the second set of characteristics includes at least a second speed to access data from the second tier of object storage devices.

10. The computer-implemented method as recited in claim 9 , wherein the first speed is greater than the second speed.

11. The computer-implemented method as recited in claim 8 , wherein the benefit is that a cost of storing the first file system object on the second tier of object storage devices is less that a cost of storing the first file system object on the first tier of object storage devices.

12. The computer-implemented method as recited in claim 8 , wherein a second metric indicates that the second file system object is likely to be accessed more frequently than the first file system object.

13. The computer-implemented method as recited in claim 8 , wherein the first set of characteristics includes at least a first cost, and wherein the second set of characteristics includes at least a second cost.

14. The computer-implemented method as recited in claim 13 , wherein the second cost is less than the first cost.

15. One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more processors cause the one or more processors to:
receive a first file system object via a network;
store the first file system object in a first object storage device of a first tier of object storage devices of a plurality of tiers of object storage devices, wherein the plurality of tiers includes a second tier of object storage devices, wherein the first tier has a first set of characteristics with a first set of values, wherein the second tier has a second set of characteristics with a second set of values, and wherein the first set of values is different in at least one respect from the second set of values;
collect one or more metrics associated with the first file system object, including a first metric;
detect, based at least upon the first metric, that a condition has been met;
initiate, based at least in part on detecting that the condition has been met, a first transfer of at least a portion of contents of the first file system object from the first object storage device to a second object storage device of the second tier of object storage devices, independent of a client-initiated transfer request specific to the first file system object from a client of a file system to which the first file system object belongs, to make a first portion of the first object storage device available for use, wherein a difference between the first set of values and the second set of values p

Assignees

• Amazon Tech Inc

Inventors

• Krishnan Karthikeyan
• Parthasarathy Akshai
• Sait Abdul Sathar

Classifications

• G06F16/185 Primary

  Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof (details of archiving G06F16/11) · CPC title

• G06F16/122

  using management policies (point-in-time backing up or restoration of persistent data G06F11/1446; file migration policies for HSM systems G06F16/185) · CPC title

• G06F16/183 Primary

  Provision of network file services by network file servers, e.g. by using NFS, CIFS (network file access protocols H04L67/1097) · CPC title

• G06F3/0643

  Management of files · CPC title

• G06F3/0611

  in relation to response time · CPC title