Automatic movement of deduped data to archival tiers of cloud storage based on access patterns

Patent metadata

Field	Value
Publication number	US-12099442-B2
Application number	US-202217571252-A
Country	US
Kind code	B2
Filing date	Jan 7, 2022
Priority date	Jan 7, 2022
Publication date	Sep 24, 2024
Grant date	Sep 24, 2024

Abstract

A method, apparatus, and system manages an object in a storage provider that provides a plurality of storage classes of storage. The method may include receiving a request for transfer of the object comprising segments from a first storage class to a second storage class of the storage provider; storing metadata associated with the object; determining that each of the segments satisfies a data storage policy based on the access characteristics of each of the segments within the object, wherein the determining includes: determining that none of the segments are referenced by the new segment for a predetermined time period; determining that none of the segments are accessed for the predetermined time period from the last accessed time instance; and in response to determining that each of the segments satisfies the data storage policy, transferring the object from the first storage class to the second storage class of the storage provider.

First claim

What is claimed is:

1. A computer-implemented method for managing a plurality of objects having deduplicated data in a storage provider that provides a plurality of storage classes of storage, the method comprising:
receiving a request for transfer of the plurality of objects comprising segments from a first storage class to a second storage class of the storage provider;
storing metadata associated with the plurality of objects, the metadata indicating access characteristics of each of the segments within the plurality of objects, the access characteristics including a last referenced time instance when each of the segments was last referenced by a new segment and a last accessed time instance when each of the segments was last accessed;
marking each of the plurality of objects for transfer to the second storage class in response to determining that each of the segments satisfies a data storage policy based on the access characteristics of each of the segments within the plurality of objects;
preparing a manifest list including each of the plurality of objects based on each of the plurality of objects being marked for transfer;
utilizing the manifest list to invoke a batch operation to transfer each of the plurality of objects from the first storage class to the second storage class of the storage provider;
generating a new copy of each of the segments in a new object in the first storage class in response to each of the segments within the second storage class being referenced by a second new segment;
determining whether the last referenced time instance to when each of the segments is referenced by the second new segment exceeds a predetermined threshold;
in response to determining that the new copy of each of the segments in the first storage class is already in the second storage class, indicating that the new object is not eligible to be transferred to the second storage class; and
in response to determining that the new copy of each of the segments in the first storage class is not already in the second storage class, indicating that the new object is eligible to be moved to the second storage class.

2. The method of claim 1 , wherein determining that each of the segments satisfies the data storage policy based on the access characteristics of each of the segments within the plurality of objects comprises:
determining that none of the segments are referenced by the new segment for a predetermined time period; and
determining that none of the segments are accessed for the predetermined time period from the last accessed time instance.

3. The method of claim 1 , wherein determining that each of the segments satisfies the data storage policy based on the access characteristics of each of the segments within the plurality of objects comprises:
determining the last referenced time instance when each of the segments was last referenced by the new segment; and
determining the last accessed time instance when each of the segments was last accessed.

4. The method of claim 1 , wherein marking each of the plurality of objects for transfer to the second storage class of the storage provider comprises:
updating a flag in one of fields in the metadata associated with each of the plurality of objects.

5. The method of claim 1 , further comprising:
performing a garbage collection (GC) operation or any independent process;
determining, during the GC operation, that the last referenced time instance when each of the segments was last referenced by the new segment exceeds a predetermined threshold;
determining, during the GC operation, that the last accessed time instance when each of the segments was last accessed exceeds the predetermined threshold; and
marking, during the GC operation, each of the plurality of objects for transfer to the second storage class of the storage provider.

6. The method of claim 1 , further comprising:
for each object included in the manifest list:
issuing an application programming interface (API) call to transfer the object from the first storage class to the second storage class of the storage provider; and
updating the metadata of the object to indicate that the object is located in the second storage class of the storage provider.

7. The method of claim 1 , further comprising:
adding counters for each segment;
updating a respective counter associated with each segment, when any segment is referenced by the new segments and/or is accessed for read operations;
determining whether any segment is referenced by the new segments within a predetermined threshold;
in response to determining that any segment is referenced by the new segments within the predetermined threshold, determining whether any segment is accessed for the read operations within the predetermined threshold; and
in response to determining that any segment is not referenced by the new segments and not accessed for the read operations within the predetermined threshold, copying the segment to a new object, wherein the new object is going to be moved to the second storage class.

8. The method of claim 1 , wherein a region comprises the segments of a first object of the plurality of objects, the method further comprising:
adding counters for the region comprising the segments;
updating a respective counter associated with each region, when any segment within the region is referenced by the new segments and/or is accessed for read operations;
determining whether any segment within the region is referenced by the new segments within a predetermined threshold;
in response to determining that any segment within the region is referenced by the new segments within the predetermined threshold, determining whether any segment within the region is accessed for the read operations within the predetermined threshold; and
in response to determining that any segment within the region is not referenced by the new segments and not accessed for the read operations within the predetermined threshold, copying the region to a new object, wherein the new object is going to be moved to the second storage class.

9. The method of claim 1 , further comprising:
adding counters for a first object of the plurality of objects;
updating a respective counter associated with each segment, when any segment within the first object is referenced by the new segments and/or is accessed for read operations;
determining whether any segment within the first object is referenced by the new segments within a predetermined threshold;
in response to determining that any segment within the first object is referenced by the new segments within the predetermined threshold, determining whether any segment within the first object is accessed for the read operations within the predetermined threshold; and
in response to determining that any segment within the first object is not referenced by the new segments and not accessed for the read operations within the predetermined threshold, indicating that the first object is eligible to be moved to the second storage class.

10. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations of managing a plurality of objects having deduplicated data in a storage provider that provides a plurality of storage classes of storage, the operations comprising:
receiving a request for transfer of the plurality of objects comprising segments from a first storage class to a second storage class of the storage provider;
storing metadata associated with the plurality of objects, the metadata indicating access characteristics of each of the segments within the plurality of objects, the access characteristics inclu

Assignees

• Dell Products Lp

Inventors

• Rath Jagannathdas
• Gunda Kalyan C

Classifications

• G06F3/0647

  Migration mechanisms · CPC title

• G06F3/0604

  Improving or facilitating administration, e.g. storage management · CPC title

• G06F3/0679

  Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP] · CPC title

• G06F3/061

  Improving I/O performance · CPC title

• G06F3/0685

  Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays · CPC title