Methods and systems of verifying data synchronized to multiple data stores

Patent metadata

Field	Value
Publication number	US-12423294-B2
Application number	US-202318176772-A
Country	US
Kind code	B2
Filing date	Mar 1, 2023
Priority date	Mar 1, 2023
Publication date	Sep 23, 2025
Grant date	Sep 23, 2025

Abstract

A data verification agent serves to verify data that is being redundantly stored in both a first data store and a second data store. The data verification agent compares corresponding objects in the first and second data stores to determine any mismatch between the objects. The data verification agent includes a mismatch ignore function causing the agent to skip a comparison of corresponding data objects in the first and second data stores in response to an indication that a mismatch between the objects would not necessarily indicate a synchronization failure so as to minimize false positive determinations of a synchronization failure.

First claim

What is claimed is:

1. A method of verifying data being redundantly stored in both a first data store and a second data store while limiting false positive indications of data mismatch, the method comprising:
synchronizing a plurality of data objects stored in the first and second data stores;
based on the plurality of data objects, determining, via a data verification agent, that the first and second data stores are each organized by different schema;
in response to the determination that the first and second data stores are each organized by different schema, applying a mapping layer to produce mapped objects based on data and properties of the data objects from the first and second data stores;
identifying, via the data verification agent, a plurality of corresponding mapped objects from the first and second data stores, wherein the corresponding mapped objects have data object properties in one schema with equivalent counterparts in another schema;
removing, by the data verification agent, mismatches caused by the first and second data stores being organized by different schema, wherein mismatches are removed when the mismatches are based on the equivalent counterparts in the one schema not being in the another schema; and
for each pair of the corresponding mapped objects:
determining whether the pair of corresponding mapped objects has a condition indicating that a data mismatch between the objects of the pair is indicative of a data synchronization failure;
in response to determining that the pair has the condition indicating that the data mismatch between the objects of the pair is indicative of the data synchronization failure:
comparing the corresponding mapped objects for data mismatch;
based on the comparison, identifying a data mismatch between the compared objects; and
in response to identifying the data mismatch between the compared objects, correcting the data mismatch based on the comparison.

2. The method of claim 1 , wherein the data verification agent is executing in an electronic environment for conducting verification of synchronization between corresponding objects stored, respectively, in the first and second data stores.

3. The method of claim 1 , further comprising skipping comparison of the objects of a pair of corresponding mapped objects for data mismatch if the pair does not have the condition indicating that a data mismatch between the objects of the pair is indicative of the data synchronization failure.

4. A system for verifying data that is redundantly stored in both a first data store and a second data store, the system comprising:
a processor;
a memory comprising programming instructions for execution by the processor; and
an interface for connecting to multiple data stores;
the programming instructions, when executed by the processor, causing the system to perform operations comprising:
synchronizing a plurality of data objects stored in the first and second data stores;
based on the plurality of data objects, determining, via a data verification agent, that the first and second data stores are each organized by different schema;
in response to the determination that the first and second data stores are each organized by different schema, applying a mapping layer to produce mapped objects based on data and properties of the data objects from the first and second data stores;
identifying, via the data verification agent, a plurality of corresponding mapped objects from the first and second data stores, wherein the corresponding mapped objects have data object properties in one schema with equivalent counterparts in another schema;
removing, by the data verification agent, mismatches caused by the first and second data stores being organized by different schema, wherein mismatches are removed when the mismatches are based on the equivalent counterparts in the one schema not being in the another schema; and
for each pair of the corresponding mapped objects:
determining whether the pair of corresponding mapped objects has a condition indicating that a data mismatch between the objects of the pair is indicative of a data synchronization failure;
in response to determining that the pair has the condition indicating that the data mismatch between the objects of the pair is indicative of the data synchronization failure:
comparing the corresponding mapped objects for data mismatch;
based on the comparison, identifying a data mismatch between the compared objects; and

in response to identifying the data mismatch between the compared objects, correcting the data mismatch based on the comparison.

5. The system of claim 4 , further comprising programming instructions that, when executed by the processor, cause the system to perform operations comprising generating an indication that the data mismatch would not indicate a synchronization failure based on a changedTime parameter of the data objects.

6. The system of claim 4 , further comprising programming instructions that, when executed by the processor, cause the system to perform operations comprising:
receiving a selection of a specific service for which to verify data synchronization;
sampling corresponding objects in the first and second data stores that have a timestamp by the selected service within a specified time range; and
reporting mismatches between the objects of the first and second data stores that have a timestamp by the selected service within the specified time range.

7. The system of claim 4 , further comprising programming instructions that, when executed by the processor, cause the system to perform operations comprising:
comparing a tombstone object from the first data store with a corresponding empty result from the second data store; and
reporting a mismatch if the second data store does not return the empty result corresponding to the tombstone object from the first data store.

8. The system of claim 4 , further comprising programming instructions that, when executed by the processor, cause the system to perform operations comprising performing data verification in a series of iterations, wherein a list of sensitive data objects are sampled for possible mismatches between the first and second data stores in each of the iterations.

9. A non-transitory computer-readable medium comprising instructions for verifying data redundantly stored in both a first data store and a second data store, the instructions, when executed by a processor, cause a computing device to perform operations comprising:
synchronizing a plurality of data objects stored in first and second data stores;
based on the plurality of data objects, determining, via a data verification agent, that the first and second data stores are each organized by different schema;
in response to the determination that the first and second data stores are each organized by different schema, applying a mapping layer to produce mapped objects based on data and properties of the data objects from the first and second data stores;
identifying, via the data verification agent, a plurality of corresponding mapped objects from the first and second data stores, wherein the corresponding mapped objects have data object properties in one schema with equivalent counterparts in another schema;
removing, by the data verification agent, mismatches caused by the first and second data stores being organized by different schema, wherein mismatches are removed when the mismatches are based on the equivalent counterparts in the one schema not being in the another schema; and
for each pair of the corresponding mapped objects:
determining whether the pair of corresponding mapped objects has a condition indicating that a data mismatch between the objects of the pai

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Pu Shi
• Kasetty Shruti
• Agarwal Anish
• Xie Hao
• Kadian Abhishek
• Gadriwala Umme Salma Shabbir
• Larsson Carly

Classifications

• G06F16/27

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title

• G06F11/1662

  the resynchronized component or unit being a persistent storage device (re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008) · CPC title

• G06F3/0619

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F11/2094

  Redundant storage or storage space (G06F11/2056 takes precedence) · CPC title

• G06F11/2074

  Asynchronous techniques · CPC title