Object storage system with meta object replication

Patent metadata

Field	Value
Publication number	US-11074002-B2
Application number	US-201916447291-A
Country	US
Kind code	B2
Filing date	Jun 20, 2019
Priority date	Jun 20, 2019
Publication date	Jul 27, 2021
Grant date	Jul 27, 2021

Abstract

Example object storage systems, meta object generators, and methods provide meta objects for replication of configuration data between data object stores. A meta object may be generated that includes configuration data. A meta object identifier may be assigned to the meta object. The meta object may be stored in a first object store with related data objects and may be replicated to a second object data store with the related data objects.

First claim

What is claimed is:

1. A system, comprising:
a first object data store configured to:
store user data objects, wherein each user data object includes user object data stored in a storage medium of the first object data store; and
associate, in a first metadata store, a storage location for the user object data for each user data object with an object identifier and object metadata for that user data object;

the first metadata store configured to:
store the object identifiers and object metadata for accessing user data objects stored in the first object data store; and
store configuration data for the first object data store, wherein the configuration data includes configuration parameters for rebuilding the configuration data in a recovery metadata store;

a storage interface configured to read the configuration data from the first metadata store to define a storage protocol for processing object data requests for a related set of data objects stored in the first object data store;
a meta object generator configured to:
generate a meta object, wherein:
the meta object is a data object containing object data; and
the object data includes meta object data selected from the configuration parameters;

assign a meta object identifier to the meta object;
store the meta object with the related set of data objects in the first object data store; and
associate, in the first metadata store, a storage location for the meta object data with the meta object identifier and object metadata for the meta object;

a replication engine configured to replicate, to a second object data store:
a plurality of data objects that include the meta object and the related set of data objects;
the meta object identifier and object metadata for the meta object; and
the object identifiers and object metadata for the related set of data objects; and

a disaster recovery manager configured to:
determine a recovery object data store and the recovery metadata store;
recover the meta object from the second object data store to the recovery object data store;
rebuild, based on the configuration parameters in the meta object, configuration data in the recovery metadata store, wherein the storage interface is further configured to read the rebuilt configuration data from the recovery metadata store to define the storage protocol for processing object data requests for the related set of data objects stored in the recovery object data store; and
recover the related set of data objects from the second object data store to the recovery object data store.

2. The system of claim 1 , wherein:
the first object data store is configured to store the plurality of data objects in a data object bucket;
the meta object data includes bucket configuration data; and
the related set of data objects include data objects stored in the data object bucket.

3. The system of claim 1 , wherein the meta object identifier includes:
a globally unique identifier for the meta object; and
a meta tag appended to a key value associated with the globally unique identifier.

4. The system of claim 1 , wherein:
the meta object generator is further configured to generate a hidden header value;
the meta object identifier includes the hidden header value; and
the first object data store is further configured to require the hidden header value to access the meta object.

5. The system of claim 1 , wherein the meta object generator is further configured to encrypt the meta object data in the meta object.

6. The system of claim 1 , wherein:
the replication engine is further configured to replicate the plurality of data objects from the first object data store to the second object data store in a priority order; and
the meta object has a higher priority than the related set of data objects.

7. The system of claim 1 , further comprising:
a meta object update handler configured to:
detect changed configuration data in the configuration data; and
initiate the meta object generator to generate an updated meta object including the changed configuration data.

8. The system of claim 7 , wherein the meta object update handler is further configured to periodically initiate the meta object generator at predetermined intervals.

9. The system of claim 1 , wherein the disaster recovery manager is further configured to:
initialize a new data bucket in the recovery object data store; and
rebuild, based on the configuration parameters in the meta object, bucket metadata for the new data bucket in the recovery metadata store.

10. A computer-implemented method, comprising:
storing, in a first metadata store, configuration data for a first object data store, wherein the configuration data includes configuration parameters for rebuilding the configuration data in a recovery metadata store;
storing user data objects in the first object data store, wherein each user data object includes user object data stored in a storage medium of the first object data store;
associating, in the first metadata store, a storage location for the user object data for each user data object with an object identifier and object metadata for that user data object;
storing, in the first metadata store, the object identifiers and object metadata for accessing user data objects stored in the first object data store;
reading configuration data from the first metadata store to define a storage protocol for processing object data requests for a related set of data objects stored in the first object data store;
generating a meta object, wherein:
the meta object is a data object containing object data;
the object data includes meta object data selected from the configuration data; and
the selected configuration data includes configuration parameters for rebuilding the configuration data in the recovery metadata store;

assigning a meta object identifier to the meta object;
storing the meta object with a related set of data objects in the first object data store;
associating, in the first metadata store, a storage location for the meta object data with the meta object identifier and object metadata for the meta object;
replicating, to a second object data store:
a plurality of data objects that includes the meta object and the related set of data objects;
the meta object identifier and object metadata for the meta object; and
the object identifiers and object metadata for the related set of data objects;

determining a recovery object data store and the recovery metadata store;
recovering the meta object from the second object data store to the recovery object data store;
rebuilding, based on the configuration parameters in the meta object, configuration data in the recovery metadata store;
recovering the related set of data objects from the second object data store to the recovery object data store; and
reading the rebuilt configuration data from the recovery metadata store to define the storage protocol for processing object data requests for the related set of data objects stored in the recovery object data store.

11. The computer-implemented method of claim 10 , further comprising:
configuring the first object data store to store the plurality of data objects in a data object bucket, wherein:
the meta object data is bucket configuration data; and
the related set of data objects are data objects stored in the data object bucket.

12. The computer-implemented method of claim 10 , wherein the meta object identifier includes:
a globally unique identifier for the meta object; and
a meta tag appended to a key value associated with the globally unique identifier.

13. The computer-implemen

Assignees

• Western Digital Tech Inc

Inventors

• Pyati Ameet
• Patale Vibhor

Classifications

• G06F3/0629

  Configuration or reconfiguration of storage systems · CPC title

• G06F3/0644

  Management of space entities, e.g. partitions, extents, pools · CPC title

• G06F3/067 Primary

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/0604

  Improving or facilitating administration, e.g. storage management · CPC title

• G06F3/065 Primary

  Replication mechanisms · CPC title