Incremental vault to object store

Patent metadata

Field	Value
Publication number	US-11403178-B2
Application number	US-201816148887-A
Country	US
Kind code	B2
Filing date	Oct 1, 2018
Priority date	Sep 29, 2017
Publication date	Aug 2, 2022
Grant date	Aug 2, 2022

Abstract

Systems and methods for managing incremental data backups on an object store. A computing device receives first data representing a changed chunk of data in a revision of a data volume on a storage device, the changed chunk includes data having changes from previous data of a previous revision. The computing device creates a block of data representing a copy of the changed chunk on the object store, the object store also includes a previous revision block representing previous revision data. The computing device determines a previous index stored on the object store corresponding to the previous revision, which includes entries including at least one corresponding to the previous revision block. The computing device creates a copy of at least one previous index from the object store, and a revised index that updates the corresponding entry with updated entry data representing the change block.

First claim

The invention claimed is:

1. A method of managing incremental data backups on an object store, the method comprising:
receiving, by a computing device, first data representing a changed chunk of data in a revision of a data volume stored on a storage device, the changed chunk of data comprising data having changes from previous data of a previous revision of the data volume, wherein the changed chunk of data is stored on the storage device in a format associated with the storage device;
creating, by the computing device, a changed block of data representing a copy of the changed chunk of data for storage on the object store as object data, wherein the object store further stores a previous revision block of data representing previous revision data of the data volume stored as object data;
determining, by the computing device, a previous index stored on the object store corresponding to the previous revision of the data volume, the previous index comprising at least one index page having entries, at least one entry corresponding to the previous revision block of data stored on the object store;
creating, by the computing device, a copy of at least one previous index page of the previous index from the object store;
creating, by the computing device, a revised index corresponding to the revision of the data volume, wherein creating the revised index comprises:
determining the entry in the revised index corresponding to the previous revision block of data; and
updating the entry with updated entry data representing the changed block of data, such that the revision of the data volume is retrievable from the object store using the revised index;

receiving, by the computing device, a request to delete the revision of the data volume stored on the object store;
retrieving, by the computing device, a set of indexes from a plurality of indexes, the set of indexes consisting of the revised index, the previous index, and a subsequent index corresponding to a subsequent revision of the data volume on the storage device, wherein a quantity of indexes within the set of indexes is less than a quantity of indexes within the plurality of indexes; and
for each respective entry in the revised index:
determining, from the retrieved set of indexes, whether the respective entry from the previous index and the revised index are different;
determining, from the retrieved set of indexes, whether the respective entry from the revised index and the subsequent index are different; and
when the respective entry from the previous index and the revised index are different and when the respective entry from the revised index and the subsequent index are different, requesting, by the computing device, the object store to delete a corresponding data block referenced by the respective entry in the revised index.

2. The method of claim 1 , wherein the changed block of data is a compressed version of the changed chunk of data.

3. The method of claim 1 , wherein creating the revised index further comprises storing, by the computing device, the revised index in the object store.

4. The method of claim 1 , wherein index entries in the revised index, corresponding to unchanged blocks of data representing chunks of data in the revision of the data volume that are unchanged since the previous revision, are the same as corresponding index entries in the previous index.

5. The method of claim 1 , further comprising restoring the revision of the data volume stored on the object store to the storage device, wherein restoring the revision comprises:
receiving, by the computing device, a request to restore the revision of the data volume stored on the object store to the storage device;
retrieving, by the computing device, the revised index corresponding to the revision of the data volume and an earlier index corresponding to an existing revision of the data volume on the storage device;
identifying, by the computing device, a set of changed blocks stored on object store corresponding to entries in the revised index, the set of changed blocks being blocks of data representing data that that have changed since the existing revision of the data volume on the storage device; and
retrieving, by the computing device, the set of changed blocks.

6. The method of claim 5 , wherein the set of changed blocks corresponds to index entries in the revised index that differ from corresponding index entries in the earlier index.

7. The method of claim 6 , further comprising copying the set of changed blocks to the storage device thereby restoring the storage device to the revision of the data volume.

8. The method of claim 6 , further comprising mounting requested blocks of the set of changed blocks such that they are accessible by the storage device, thereby accomplishing an on-demand restore of the revision of the data volume.

9. The method of claim 1 , further comprising calculating a space reclamation amount of a delete of the revision of the data volume stored on the object store to the storage device, wherein calculating a space reclamation amount comprises:
receiving, by the computing device, a request to calculate the space reclamation amount of a delete of the revision of the data volume stored on the object store;
retrieving, by the computing device, the revised index, the previous index, and the subsequent index corresponding to the subsequent revision of the data volume on the storage device;
determining, by the computing device, deletable blocks of data by comparing index entries in the revised index to corresponding index entries in the previous index and the subsequent index, the deletable blocks of data having corresponding index entries in the revised index that differ from corresponding data entries in the previous index and the subsequent index;
calculating a size of each of the deletable blocks of data;
calculating a size of metadata corresponding to the deletable blocks of data; and
calculating the space reclamation amount by adding the sizes of each of the deletable blocks of data and the metadata.

10. The method of claim 1 , further comprising updating a retention policy for data blocks in the object store corresponding to entries in the revised index.

11. The method of claim 10 , wherein the retention policy is write once read many (WORM) protection.

12. A computerized system for managing incremental data backups on an object store, the system comprising a processor configured to run a module stored in memory that is configured to cause the processor to:
receive first data representing a changed chunk of data in a revision of a data volume stored on a storage device, the changed chunk of data comprising data having changes from previous data of a previous revision of the data volume, wherein the changed chunk of data is stored on the storage device in a format associated with the storage device;
create a changed block of data representing a copy of the changed chunk of data for storage on the object store as object data, wherein the object store further stores a previous revision block of data representing previous revision data of the data volume stored as object data;
determine a previous index stored on the object store corresponding to the previous revision of the data volume, the previous index comprising at least one index page having entries, at least one entry corresponding to the previous revision block of data stored on the object store;
create a copy of at least one previous index page of the previous index from the object store;
create a revised index corresponding to the revision of the data volume, wherein the step of creating the revised index comprises:
determine the entry

Assignees

• Google Llc

Inventors

• Murphy Christopher

Classifications

• G06F11/1451 Primary

  by selection of backup contents · CPC title

• G06F2201/80

  Database-specific techniques · CPC title

• G06F16/219

  Managing data history or versioning (querying versioned data G06F16/2474; querying temporal data G06F16/2477) · CPC title

• G06F16/27 Primary

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title

• G06F2201/805

  Real-time · CPC title