Efficient cleanup/defragmentation mechanism for expired retention locked (compliance and governance) segments in deduped cloud objects

Patent metadata

Field	Value
Publication number	US-11971785-B2
Application number	US-202017071529-A
Country	US
Kind code	B2
Filing date	Oct 15, 2020
Priority date	Oct 15, 2020
Publication date	Apr 30, 2024
Grant date	Apr 30, 2024

Abstract

One example method includes identifying a cloud object as a potential candidate for defragmentation, evaluating the cloud object to determine what portion of segments of the cloud object are expired, when the portion of expired segments meets or exceeds a threshold, segregating the expired segments and unexpired segments of the cloud object, creating a first new cloud object that includes only unexpired segments, creating a second new cloud object that includes only expired segments, and deleting the cloud object from storage.

First claim

What is claimed is:

1. A method, comprising:
identifying a cloud object as a potential candidate for defragmentation in a cloud storage;
evaluating the cloud object to determine what portion of segments of the cloud object are expired; and
when the portion of expired segments meets or exceeds a threshold:
segregating the expired segments and unexpired segments of the cloud object;
creating in the cloud storage a plurality of first new cloud objects, which include only unexpired segments, based on expiry times of the unexpired segments;
creating in the cloud storage a second new cloud object that includes only expired segments, which have not been deleted; and
deleting the cloud object from the cloud storage,

wherein each of the plurality of first new cloud objects is locked during a respective retention duration, which is based on expiration durations of segments included in each first new cloud object.

2. The method as recited in claim 1 , wherein the cloud object is a deduplicated object.

3. The method as recited in claim 1 , wherein the plurality of first new cloud objects are locked, and the second new cloud object is not locked.

4. The method as recited in claim 1 , wherein the plurality of first new cloud objects are zoned so that all segments of the plurality of first new cloud objects have respective expiration times that have not yet come and fall within a specified time frame.

5. The method as recited in claim 1 , wherein the respective retention duration is a longest expiration duration among segments saved in a respective first new cloud object.

6. The method as recited in claim 1 , wherein the respective retention duration is a shortest expiration duration among segments saved in a respective first new cloud object.

7. The method as recited in claim 6 , wherein the plurality of first new cloud objects will expire at an end of the shortest expiration duration that has not yet come and prior to expiration of one of the plurality of first new cloud objects, the method further comprises:
copying forward all segments whose respective expiration duration is longer than the shortest expiration duration;
using the copy forwarded segments to create a second new cloud object; and
deleting the plurality of first new cloud objects from the cloud storage.

8. The method as recited in claim 1 , further comprising performing a liveness check, and deleting the second new cloud object based on results of the liveness check.

9. The method as recited in claim 1 , wherein segregating the expired segments and unexpired segments comprises copying the unexpired segments into a first in-memory buffer, and copying the expired segments into a second in-memory buffer.

10. The method as recited in claim 1 , further comprising:
receiving a lock request concerning the cloud object;
determining whether the cloud object is locked or not locked;
when the cloud object is determined to not be locked, locking the cloud object according to a duration of a segment of the cloud object that has a shortest duration of all segments of the cloud object, and updating metadata of the cloud object and metadata of the segments; and
when the cloud object is determined to be locked, determining whether a new lock duration specified in the lock request is more or less than a current lock duration of the cloud object and, either:
when the new lock duration is greater than the current lock duration, updating metadata of the segments and updating a lock count of the cloud object, but not updating an expiry date of the cloud object; or
when the new lock duration is less than the current lock duration, updating only lock counts for the cloud object and for the segments.

11. A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
identifying a cloud object as a potential candidate for defragmentation in a cloud storage;
evaluating the cloud object to determine what portion of segments of the cloud object are expired; and
when the portion of expired segments meets or exceeds a threshold:
segregating the expired segments and unexpired segments of the cloud object;
creating in the cloud storage a plurality of first new cloud objects, which include only unexpired segments, based on expiry times of the unexpired segments;
creating in the cloud storage a second new cloud object that includes only expired segments, which have not been deleted; and
deleting the cloud object from the cloud storage,

wherein each of the plurality of first new cloud objects is locked during a respective retention duration, which is based on expiration durations of segments included in each first new cloud object.

12. The non-transitory storage medium as recited in claim 11 , wherein the cloud object is a deduplicated object.

13. The non-transitory storage medium as recited in claim 11 , wherein the plurality of first new cloud objects are locked, and the second new cloud object is not locked.

14. The non-transitory storage medium as recited in claim 11 , wherein the plurality of first new cloud objects are zoned so that all segments of the first new cloud object have respective expiration times that have not yet come and fall within a specified time frame.

15. The non-transitory storage medium as recited in claim 11 , wherein the respective retention duration is a longest expiration duration among segments included in a respective first new cloud object.

16. The non-transitory storage medium as recited in claim 11 , wherein the respective retention duration is a shortest expiration duration among segments included in a respective first new cloud object.

17. The non-transitory storage medium as recited in claim 16 , wherein the plurality of first new cloud objects will expire at an end of the shortest expiration duration that has not yet come, and, prior to expiration of one of the plurality of first new cloud objects, and the operations further comprise:
copying forward all segments whose respective expiration duration is longer than the shortest expiration duration;
using the copy forwarded segments to create a second new cloud object; and
deleting the plurality of first new cloud objects from the cloud storage.

18. The non-transitory storage medium as recited in claim 11 , wherein the operations further comprise performing a liveness check, and deleting the second new cloud object based on results of the liveness check.

19. The non-transitory storage medium as recited in claim 11 , wherein segregating the expired segments and unexpired segments comprises copying the unexpired segments into a first in-memory buffer, and copying the expired segments into a second in-memory buffer.

20. The non-transitory storage medium as recited in claim 11 , wherein the operations further comprise:
receiving a lock request concerning the cloud object;
determining whether the cloud object is locked or not locked;
when the cloud object is determined to not be locked, locking the cloud object according to a duration of a segment of the cloud object that has a shortest duration of all segments of the cloud object, and updating metadata of the cloud object and metadata of the segments; and
when the cloud object is determined to be locked, determining whether a new lock duration specified in the lock request is more or less than a current lock duration of the cloud object and, either:
when the new lock duration is greater than the current lock duration, updating metada

Assignees

• Emc Ip Holding Co Llc

Inventors

• Rath Jagannathdas
• Gunda Kalyan C

Classifications

• G06F11/1453 Primary

  using de-duplication of the data · CPC title

• G06F3/0619

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F3/0641

  De-duplication techniques · CPC title

• G06F3/0656

  Data buffering arrangements · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title