Proactive optimizations at multi-tier file systems

Patent metadata

Field	Value
Publication number	US-2019079940-A1
Application number	US-201816185416-A
Country	US
Kind code	A1
Filing date	Nov 9, 2018
Priority date	Mar 27, 2015
Publication date	Mar 14, 2019
Grant date	—

Abstract

A recommendations manager (RM) of a file system service identifies a file system object group (FSOG) for which proactive placement recommendations are to be generated using statistical analyses. Usage metrics collected from the FSOG are used to train a model. Predictions obtained from the model are used to generate a recommendation to transfer a file system object proactively from one storage device group of the service to another. The recommendations are provided to an object migrator of the service to initiate the transfer.

First claim

1 - 20 . (canceled)

21 . A system, comprising:
a recommendations manager of a storage service, implemented at one or more computing devices, wherein the recommendations manager is configured to:
train one or more machine learning models, including a first machine learning model, using a first collection of usage metrics of a first storage object group;
generate, based at least in part on one or more predictions obtained from the first machine learning model, a particular proactive placement recommendation to transfer a particular storage object of the first storage object group from a first storage device group of the storage service to a different storage device group; and
transmit the particular proactive placement recommendation to an object migrator component of the storage service configured to initiate the transfer of the particular storage object.

22 . The system as recited in claim 21 , wherein the recommendations manager is further configured to:
determine a first constraint on machine learning resources of a provider network to be used to generate one or more proactive placement recommendations for the first storage object group, including the particular proactive placement recommendation, wherein the training of the first machine learning model is performed using resources selected in accordance with the first constraint.

23 . The system as recited in claim 22 , wherein the first constraint is based at least in part on one or more of: (a) an identity of a client on whose behalf the first storage object group is created, (b) a size of the first storage object group, (c) an analysis of billing records associated with the first storage object group, (d) an available computing capacity at a machine learning service implemented at the provider network.

24 . The system as recited in claim 22 , wherein the recommendations manager is further configured to:
generate an additional set of proactive placement recommendations for the first storage object group using a set of resources selected in accordance with a different constraint, wherein the different constraint is determined by the recommendations manager subsequent to determining the first constraint.

25 . The system as recited in claim 21 , wherein the first machine learning model implements a first modeling methodology selected from a methodology set comprising: (a) regression, (b) classification or (c) time series modeling.

26 . The system as recited in claim 21 , wherein the recommendations manager is further configured to:
classify, using a clustering model, users of the first storage object group into a plurality of categories based at least in part on respective storage access patterns of the users, wherein the particular proactive placement recommendation is based at least in part on a result of the clustering model.

27 . The system as recited in claim 21 , wherein the recommendations manager is further configured to:
receive, via a programmatic interface of the storage service, an indication from a client of one or more storage objects for which the client has opted in for proactive placement recommendations; identify, based at least in part on the one or more storage objects for which an indication was received from the client, one or more storage object groups for which respective sets of proactive placement recommendations are to be generated using one or more machine learning models, including the first storage object group.

28 . A method, comprising:
performing, by a recommendations manager (RM) of a storage service:
training one or more machine learning models, including a first machine learning model, using a first collection of usage metrics of a first storage object group;
generating, based at least in part on one or more predictions obtained from the first machine learning model, a particular proactive placement recommendation to transfer a particular storage object of the first storage object group from a first storage device group of the storage service to a different storage device group; and
transmitting the particular proactive placement recommendation to an object migrator component of the storage service configured to initiate the transfer of the particular storage object.

29 . The method as recited in claim 28 , further comprising:
determining, by the RM, a first constraint on machine learning resources of a provider network to be used for generating one or more proactive placement recommendations for the first storage object group, including the particular proactive placement recommendation, wherein the training of the first machine learning model is performed using resources selected in accordance with the first constraint.

30 . The method as recited in claim 29 , wherein the first constraint is based at least in part on one or more of: (a) an identity of a client on whose behalf the first storage object group is created, (b) a size of the first storage object group, (c) an analysis of billing records associated with the first storage object group, (d) an available computing capacity at a machine learning service implemented at a provider network.

31 . The method as recited in claim 29 , further comprising:
generating, by the RM, an additional set of proactive placement recommendations for the first storage object group using a set of resources selected in accordance with a different constraint, wherein the different constraint is determined by the recommendations manager subsequent to determining the first constraint.

32 . The method as recited in claim 28 , wherein the first machine learning model implements a first modeling methodology selected from a methodology set comprising: (a) regression, (b) classification or (c) time series modeling.

33 . The method as recited in claim 28 , further comprising performing, by the RM:
classifying, using a clustering model, users of the first storage object group into a plurality of categories based at least in part on respective storage access patterns of the users, wherein the particular proactive placement recommendation is based at least in part on a result of the clustering model.

34 . The method as recited in claim 28 , further comprising performing, by the RM:
receiving, via a programmatic interface, an indication from a client of one or more storage objects for which the client has opted in for proactive placement recommendations; identifying, based at least in part on the one or more storage objects for which an indication was received from the client, one or more storage object groups for which respective sets of proactive placement recommendations are to be generated using one or more machine learning models, including the first storage object group.

35 . One or more non-transitory computer-readable storage media storing program instructions that when executed on or across one or more processors implements a recommendations manager (RM) of a storage service, wherein the recommendations manager is configured to:
train one or more machine learning models, including a first machine learning model, using a first collection of usage metrics of a first storage object group; generate, based at least in part on one or more predictions obtained from the first machine learning model, a particular proactive placement recommendation to transfer a particular storage object of the first storage object group from a first storage device group of the storage service to a different storage device group; and transmit the particular proactive placement recommendation to an object migrator component of the storage service configured to initiate t

Assignees

• Amazon Tech Inc

Inventors

• Krishnan Karthikeyan
• Sait Abdul Sathar

Classifications

• G06N99/005

  Physics · mapped topic

• G06F17/3097

  Physics · mapped topic

• G06F17/30218

  Physics · mapped topic

• G06F17/30079 Primary

  Physics · mapped topic

• G06F16/122

  using management policies (point-in-time backing up or restoration of persistent data G06F11/1446; file migration policies for HSM systems G06F16/185) · CPC title