Object storage system with versioned meta objects

Patent metadata

Field	Value
Publication number	US-11093465-B2
Application number	US-201916447366-A
Country	US
Kind code	B2
Filing date	Jun 20, 2019
Priority date	Jun 20, 2019
Publication date	Aug 17, 2021
Grant date	Aug 17, 2021

Abstract

Example object storage systems, meta object generators, and methods provide versioned meta objects for internal operational data that may be replicated between data object stores. A meta object may be generated that includes meta object data, such as internal operational data. A meta object identifier may be assigned to the meta object. A first version identifier may be associated with the meta object data and a second version identifier may be associated with a marker for the meta object, where the marker prevents exposure as a user data object.

First claim

What is claimed is:

1. A system, comprising:
a first object data store configured to:
store a plurality of data objects, wherein each data object includes object data stored in a storage medium of the first object data store; and
associate, in a first metadata store, a storage location for the object data for each data object with an object identifier and object metadata for that data object;

the first metadata store configured to:
store the object identifiers and object metadata for accessing data objects stored in the first object data store; and
store configuration data for the first object data store, wherein the configuration data includes configuration parameters for rebuilding the configuration data in a recovery metadata store;

a meta object generator configured to:
generate a first version of a meta object, wherein the first version of the meta object includes:
meta object data selected from the configuration parameters;
a meta object identifier assigned to the meta object; and
a first version identifier associated with the first version of the meta object data;

generate a second version of the meta object, wherein the second version of the meta object includes:
the meta object identifier assigned to the meta object; and
a second version identifier associated with a marker for the meta object, wherein the marker prevents exposure of the meta object as a user data object;

store each version of the meta object as a data object in the first object data store with the plurality of data objects; and
associate, in the first metadata store, at least one storage location for the meta object data with the meta object identifier and object metadata for each version of the meta object; and

a storage interface configured to:
read the configuration data from the first metadata store to define a storage protocol for processing object data requests for the plurality of data objects stored in the first object data store; and
read rebuilt configuration data from the recovery metadata store, wherein the rebuilt configuration data is based on the configuration parameters from the first version of the meta object.

2. The system of claim 1 , wherein:
the meta object generator is further configured to:
generate the second version of the meta object prior to generating the first version of the meta object; and
generate a greater version value for the second version identifier of the meta object than a first version value for the first version identifier of the meta object; and

the storage interface is further configured to determine a current version of the meta object based on a greatest version value.

3. The system of claim 2 , wherein:
the first object data store is further configured to store the plurality of data objects in a data object bucket;
the meta object data includes bucket configuration parameters for the data object bucket; and
the meta object and user data objects are stored in the data object bucket.

4. The system of claim 1 , further comprising:
a replication manager configured to replicate the meta object and user data objects from the first object data store to a second object data store.

5. The system of claim 1 , wherein the meta object generator is further configured to use a version control function to determine the first version identifier and the second version identifier.

6. The system of claim 5 , wherein the version control function for the second version identifier is configured to increase a timestamp value for the marker to determine the second version identifier.

7. The system of claim 1 , wherein the marker is a delete marker.

8. The system of claim 1 , wherein the meta object generator is further configured to:
receive a configuration change for updated configuration data for the first object data store;
generate an updated meta object, wherein the updated meta object includes:
updated meta object data based on the updated configuration data;
the meta object identifier assigned to the meta object;
a third version identifier associated with the updated meta object data; and
a fourth version identifier associated with an updated marker for the meta object, wherein the updated marker prevents exposure as a user data object; and

store the updated meta object in the first object data store.

9. The system of claim 1 , wherein:
the storage interface is further configured to:
receive a user data request including the meta object identifier; and
return, responsive to the marker, an error message.

10. A computer-implemented method, comprising:
storing, in a first object data store, a plurality of data objects, wherein each data object includes object data stored in a storage medium of the first object data store;
storing, in a first metadata store, a storage location for the object data for each data object with an object identifier and object metadata for that data object;
storing, in the first metadata store, configuration data for the first object data store, wherein the configuration data-includes configuration parameters for rebuilding the configuration data in a recovery metadata store;
selecting meta object data for a meta object from the configuration parameters;
assigning a meta object identifier to the meta object;
generating a first version of the meta object including the meta object data;
assigning a first version identifier to the first version of the meta object data;
generating a second version of the meta object;
assigning, to the second version of the meta object, a second version identifier associated with a marker for the meta object, wherein the marker prevents exposure of the meta object as a user data object;
storing each version of the meta object as a data object in the first object data store;
associating, in the first metadata store, at least one storage location for the meta object data with the meta object identifier and object metadata for each version of the meta object;
reading the configuration data from the first metadata store to define a storage protocol for processing object data requests for the plurality of data objects stored in the first object data store; and
reading rebuilt configuration data from the recovery metadata store, wherein the rebuilt configuration data is based on the configuration parameters from the first version of the meta object.

11. The computer-implemented method of claim 10 , further comprising:
generating the second version of the meta object prior to generating the first version of the meta object;
generating, for the second version identifier of the meta object, a greater version value than a first version value for the first version identifier of the meta object; and
determining a current version of the meta object based on a greatest version value.

12. The computer-implemented method of claim 11 , further comprising:
configuring the first object data store to store the plurality of data objects in a data object bucket, wherein:
the meta object data includes bucket configuration parameters for the data object bucket; and
the meta object and user data objects are stored in the data object bucket.

13. The computer-implemented method of claim 10 , further comprising:
replicating the meta object and the user data objects from the first object data store to a second object data store.

14. The computer-implemented method of claim 10 , wherein:
assigning the first version identifier uses a version control function; and
assigning the second version identifier uses the version control function.

15. The computer-implemented method of claim 14 ,

Assignees

• Western Digital Tech Inc

Inventors

• Cheru Tomy Ammuthan
• D'Halluin Carl
• Pyati Ameet

Classifications

• G06F16/27

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title

• G06F16/164

  File meta data generation · CPC title

• G06F16/219 Primary

  Managing data history or versioning (querying versioned data G06F16/2474; querying temporal data G06F16/2477) · CPC title

• G06F16/113 Primary

  Details of archiving (lifecycle management in storage systems G06F3/0649; point-in-time backing up or restoration of persistent data G06F11/1446) · CPC title