Mutable data objects content verification tool

Patent metadata

Field	Value
Publication number	US-10146600-B2
Application number	US-201615193144-A
Country	US
Kind code	B2
Filing date	Jun 27, 2016
Priority date	Dec 16, 2015
Publication date	Dec 4, 2018
Grant date	Dec 4, 2018

Abstract

Data object content verification systems and processes provide perfect reliability and low storage overhead. Object data is generated in a reproducible manner based upon object locally stored object metadata. The object data is stored to an object storage system. The stored object data is subsequently verified by retrieving the object metadata, reproducing the original object data, and comparing the stored and original object data. The data object content verification systems and processes support both mutable and immutable data object content verification.

First claim

What is claimed is:

1. A method for use with a distributed object storage system, the method comprising:
generating, in a processor, a plurality of object IDs and corresponding original object metadata;
for each object ID, generating corresponding original object data in a reproducible manner based upon the corresponding original object metadata, the original object data generated based upon a first object size;
storing the corresponding original object metadata to a metadata storage device and writing the corresponding original object data to the distributed object storage system;
selecting a plurality of the object IDs of the original object data to be mutated; and
for the object IDs to be mutated:
generating corresponding mutated object data in a reproducible manner based upon corresponding mutated object metadata, the mutated object data generated based upon a second object size that is greater than the first object size;
storing the corresponding mutated object metadata to the metadata storage device and writing the mutated object data to the distributed object storage system;
retrieving the object ID and mutated object metadata from the metadata storage device;
reproducing the mutated object data using the mutated object metadata; and
comparing the reproduced mutated object data to the retrieved mutated object metadata to identify corruption in the distributed object storage system.

2. The method of claim 1 wherein mutated object metadata comprises an updated object ID and a bitmask.

3. The method of claim 2 wherein reproducing the mutated object data using the mutated object metadata comprises:
reproducing original object data using the original object metadata;
generating updated object data using the updated object ID; and
combining the reproduced original data and the updated object data according to the bitmask to reproduce the mutated object data.

4. The method of claim 2 wherein mutated object metadata comprises an ordered list of updated object ID and bitmask tuples.

5. The method of claim 4 wherein reproducing the mutated object data using the mutated object metadata comprises:
reproducing original object data using the original object metadata;
for each tuple in the ordered list of updated object ID and bitmask tuples, iteratively generating updated object data using the updated object ID and combining the reproduced original data and the updated object data according to the bitmask to generate the reproduce the mutated object data.

6. A system for use with a distributed object storage system, the system comprising:
a content generator configured to generate object data in a reproducible manner based upon corresponding object metadata, the original object data generated based upon a first object size;
an object creator configured to:
generate a plurality of object IDs and corresponding original object metadata;
for each object ID, generate corresponding original object data using the content generator and the corresponding original object metadata;
store the corresponding original object metadata to a metadata storage device; and
write the corresponding original object data to the distributed object storage system;

an object mutator configured to:
select a plurality of the object IDs of the original object data to be mutated; and
for the object IDs to be mutated:
generate corresponding mutated object data in a reproducible manner using the content generator and the mutated object metadata, the mutated object data generated based upon a second object size that is greater than the first object size;
store the corresponding mutated object metadata to the metadata storage device; and
write the mutated object data to the distributed object storage system; and

a content verifier configure to:
for the object IDs to be mutated:
retrieve the object ID and mutated object metadata from the metadata storage device;
reproduce the mutated object data using the mutated object metadata; and
compare the reproduced mutated object data to the retrieved mutated object metadata to identify corruption in the distributed object storage system.

7. The system of claim 6 wherein mutated object metadata comprises an updated object ID and a bitmask.

8. The system of claim 7 wherein the content verifier is configured to:
reproduce original object data using the original object metadata;
generate updated object data using the updated object ID; and
combine the reproduced original data and the updated object data according to the bitmask to reproduce the mutated object data.

9. The system of claim 7 wherein mutated object metadata comprises an ordered list of updated object ID and bitmask tuples.

10. The system of claim 9 wherein the content verifier is configured to:
reproduce original object data using the original object metadata;
for each tuple in the ordered list of updated object ID and bitmask tuples, iteratively generate updated object data using the updated object ID and combine the reproduced original data and the updated object data according to the bitmask to generate the reproduce the mutated object data.

Assignees

• Emc Corp
• Emc Ip Holding Co Llc

Inventors

• Kurilov Andrey
• Danilov Mikhail
• Rakulenko Alexander
• Gusakov Kirill
• Fomin Andrey

Classifications

• G06F11/0751

  Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06) · CPC title

• G06F11/0727 Primary

  in a storage system, e.g. in a DASD or network based storage system (drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097) · CPC title

• G06F11/0709 Primary

  in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems · CPC title

• G06F11/0793

  Remedial or corrective actions (recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40) · CPC title

• G06F11/263

  Generation of test inputs, e.g. test vectors, patterns or sequences {; with adaptation of the tested hardware for testability with external testers} · CPC title