Managing overwrites when archiving data in cloud/object storage

Patent metadata

Field	Value
Publication number	US-11042504-B2
Application number	US-201816035445-A
Country	US
Kind code	B2
Filing date	Jul 13, 2018
Priority date	Jul 13, 2018
Publication date	Jun 22, 2021
Grant date	Jun 22, 2021

Abstract

Techniques for managing overwrites when archiving data in cloud/object storage are provided. In one set of embodiments, a computer system can overwrite a chunk of data residing in cloud/object storage, where the cloud/object storage only guarantees eventual consistency for overwrites, and where the overwriting is associated with a sequence number. Upon overwriting the chunk of data, the computer system create a new instance of a file in the cloud/object storage, where the new instance of the file has a version number corresponding to the sequence number.

First claim

What is claimed is:

1. A method for managing overwrites when archiving data in cloud/object storage, the method comprising:
uploading, by a computer system, a snapshot of a dataset as a plurality of data chunks to a cloud archive in the cloud/object storage, the snapshot having a snapshot number;
upon completing the uploading of the snapshot, overwriting, by the computer system, a superblock chunk in the cloud archive, wherein the superblock chunk is separate from the plurality of data chunks representing the uploaded snapshot, and wherein the overwriting comprises updating the superblock chunk with metadata associated with the uploaded snapshot; and
upon overwriting the superblock chunk, creating, by the computer system, a new instance of an archive file in the cloud archive, wherein the new instance of the archive file does not contain any data content, wherein the new instance of the archive file has a file name with a version number corresponding to the snapshot number of the uploaded snapshot, and wherein the new instance of the archive file is separate from the superblock chunk and the plurality of data chunks representing the uploaded snapshot.

2. The method of claim 1 wherein the cloud archive comprises a plurality of archived snapshots of the dataset.

3. The method of claim 1 wherein the metadata associated with the uploaded snapshot includes the snapshot number and a pointer to a root node of a metadata tree for the uploaded snapshot.

4. The method of claim 3 wherein the pointer to the root node of the metadata tree for the uploaded snapshot comprises a chunk identifier of a metadata chunk in the cloud archive.

5. The method of claim 1 wherein another computer system is configured to:
access the superblock chunk and identify a latest snapshot number included in the accessed version of the superblock chunk.

6. The method of claim 5 wherein said another computer system is further configured to:
determine whether the latest snapshot number included in the accessed version of the superblock chunk is in fact the most recent snapshot uploaded to the cloud archive by comparing the latest snapshot number with the version number in the file name of the new instance of the archive file.

7. The method of claim 5 wherein said another computer system is configured to determine that the latest snapshot number included in the accessed version of the superblock chunk is outdated if an instance of the archive file with a file name version number higher than the latest snapshot number exists in the cloud archive.

8. A non-transitory computer readable storage medium having stored thereon program code executable by a computer system, the program code embodying a method for managing overwrites when archiving data in cloud/object storage, the method comprising:
uploading a snapshot of a dataset as a plurality of data chunks to a cloud archive in the cloud/object storage, the snapshot having a snapshot number;
upon completing the uploading of the snapshot, overwriting a superblock chunk in the cloud archive, wherein the superblock chunk is separate from the plurality of data chunks representing the uploaded snapshot, and wherein the overwriting comprises updating the superblock chunk with metadata associated with the uploaded snapshot; and
upon overwriting the superblock chunk, creating a new instance of an archive file in the cloud archive, wherein the new instance of the archive file does not contain any data content, wherein the new instance of the archive file has a file name with a version number corresponding to the snapshot number of the uploaded snapshot, and wherein the new instance of the archive file is separate from the superblock chunk and the plurality of data chunks representing the uploaded snapshot.

9. The non-transitory computer readable storage medium of claim 8 wherein the cloud archive comprises a plurality of archived snapshots of the dataset.

10. The non-transitory computer readable storage medium of claim 8 wherein the metadata associated with the uploaded snapshot includes the snapshot number and a pointer to a root node of a metadata tree for the uploaded snapshot.

11. The non-transitory computer readable storage medium of claim 8 wherein another computer system is configured to:
access the superblock chunk and identify a latest snapshot number included in the accessed version of the superblock chunk.

12. The non-transitory computer readable storage medium of claim 11 wherein said another computer system is further configured to:
determine whether the latest snapshot number included in the accessed version of the superblock chunk is in fact the most recent snapshot uploaded to the cloud archive by comparing the latest snapshot number with the version number in the file name of the new instance of the archive file.

13. The non-transitory computer readable storage medium of claim 11 wherein said another computer system is configured to determine that the latest snapshot number included in the accessed version of the superblock chunk is outdated if an instance of the archive file with a file name version number higher than the latest snapshot number exists in the cloud archive.

14. A computer system comprising:
a processor; and
a non-transitory computer readable medium having stored thereon program code that, when executed, causes the processor to:
upload a snapshot of a dataset as a plurality of data chunks to a cloud archive in a cloud/object storage, the snapshot having a snapshot number;
upon completing the uploading of the snapshot, overwrite a superblock chunk in the cloud archive, wherein the superblock chunk is separate from the plurality of data chunks representing the uploaded snapshot, and wherein the overwriting comprises updating the superblock chunk with metadata associated with the uploaded snapshot; and
upon overwriting the superblock chunk, create a new instance of an archive file in the cloud archive, wherein the new instance of the archive file does not contain any data content, wherein the new instance of the archive file has a file name with a version number corresponding to the snapshot number of the uploaded snapshot, and wherein the new instance of the archive file is separate from the superblock chunk and the plurality of data chunks representing the uploaded snapshot.

15. The computer system of claim 14 wherein the cloud archive comprises a plurality of archived snapshots of the dataset.

16. The computer system of claim 14 wherein the metadata associated with the uploaded snapshot includes the snapshot number and a pointer to a root node of a metadata tree for the uploaded snapshot.

17. The computer system of claim 14 wherein another computer system is configured to:
access the superblock chunk and identify a latest snapshot number included in the accessed version of the superblock chunk.

18. The computer system of claim 17 wherein said another computer system is further configured to:
determine whether the latest snapshot number included in the accessed version of the superblock chunk is in fact the most recent snapshot uploaded to the cloud archive by comparing the latest snapshot number with the version number in the file name of the new instance of the archive file.

19. The computer system of claim 17 wherein said another computer system is configured to determine that the latest snapshot number included in the accessed version of the superblock chunk is outdated if an instance of the archive file with a file name version number higher than the latest snapshot number

Assignees

• Vmware Inc

Inventors

• Kashi Visvanathan Satish Kumar
• Sarda Pooja
• Langouev Ilya

Classifications

• G06F11/1464

  for networked environments · CPC title

• G06F2201/815

  Virtual · CPC title

• G06F16/113 Primary

  Details of archiving (lifecycle management in storage systems G06F3/0649; point-in-time backing up or restoration of persistent data G06F11/1446) · CPC title

• G06F16/1873

  Versioning file systems, temporal file systems, e.g. file system supporting different historic versions of files · CPC title

• G06F2201/84

  Using snapshots, i.e. a logical point-in-time copy of the data · CPC title