Quality analysis on object notation data sources

Patent metadata

Field	Value
Publication number	US-10324981-B2
Application number	US-201514881202-A
Country	US
Kind code	B2
Filing date	Oct 13, 2015
Priority date	Oct 13, 2015
Publication date	Jun 18, 2019
Grant date	Jun 18, 2019

Abstract

Determination of a degree of similarity among and between a set of text notation schema instances. One type of text notation schema instance is the JSON type. In some embodiments, the degree of similarity is expressed as a schema variance value which is determined by individually comparing the schema instances of the set of text notation schema instances to a representative majority schema. Also, determining a quality of a data source associated with the plurality of text notation schema instances based, at least in part, upon the similarity value.

First claim

What is claimed is:

1. A method comprising:
receiving a plurality of JSON schemas used to store some of the data in a first data store, with each JSON schema of the plurality JSON schemas respectively corresponding to a set of attributes and attribute types for parsing JSON data objects according the respectively corresponding JSON schema;
receiving a first JSON data store data set including information indicative of attribute values, attribute types and attribute organization of first plurality of JSON data objects stored in a first JSON data store;
for each given JSON schema of the plurality of JSON schemas, determining a proportion value corresponding to a proportion of the first plurality of JSON objects that conform to the given JSON schema;
determining a first majority JSON schema as a JSON schema of the first plurality of JSON schemas that has the largest respectively corresponding proportion value;
for each given JSON schema of the plurality of JSON schemas, determining a first similarity score between the given JSON schema and the first majority JSON schema, with: (i) the first similarity score of the first majority JSON schema being one, and (ii) first similarity scores for each given JSON schema other than the first majority JSON schema is based upon a degree of similarity between the given JSON schema and the first majority JSON schema;
for each given JSON schema of the plurality of JSON schemas, determining a product corresponding to the given JSON schema, with the product being a product of: (i) the proportion value of the given JSON schema, multiplied by (ii) the first similarity score of the given JSON schema;
determining a first data store variance value by summing the products respectively corresponding to the JSON schema of the plurality of JSON schemas; and
selecting the first data store to supply data based, at least in part, upon the first data store variance value.

2. A computer program product (CPP) comprising:
a machine readable storage device; and
computer code stored on the machine readable storage device, with the computer code including instructions for causing a processor(s) set to perform operations including the following:
receiving a plurality of JSON schemas used to store some of the data in a first data store, with each JSON schema of the plurality JSON schemas respectively corresponding to a set of attributes and attribute types for parsing JSON data objects according the respectively corresponding JSON schema,
receiving a first JSON data store data set including information indicative of attribute values, attribute types and attribute organization of first plurality of JSON data objects stored in a first JSON data store,
for each given JSON schema of the plurality of JSON schemas, determining a proportion value corresponding to a proportion of the first plurality of JSON objects that conform to the given JSON schema,
determining a first majority JSON schema as a JSON schema of the first plurality of JSON schemas that has the largest respectively corresponding proportion value,
for each given JSON schema of the plurality of JSON schemas, determining a first similarity score between the given JSON schema and the first majority JSON schema, with: (i) the first similarity score of the first majority JSON schema being one, and (ii) first similarity scores for each given JSON schema other than the first majority JSON schema is based upon a degree of similarity between the given JSON schema and the first majority JSON schema,
for each given JSON schema of the plurality of JSON schemas, determining a product corresponding to the given JSON schema, with the product being a product of: (i) the proportion value of the given JSON schema, multiplied by (ii) the first similarity score of the given JSON schema,
determining a first data store variance value by summing the products respectively corresponding to the JSON schema of the plurality of JSON schemas, and
selecting the first data store to supply data based, at least in part, upon the first data store variance value.

3. A computer system (CS) comprising:
a processor(s) set;
a machine readable storage device; and
computer code stored on the machine readable storage device, with the computer code including instructions for causing the processor(s) set to perform operations including the following:
receiving a plurality of JSON schemas used to store some of the data in a first data store, with each JSON schema of the plurality JSON schemas respectively corresponding to a set of attributes and attribute types for parsing JSON data objects according the respectively corresponding JSON schema,
receiving a first JSON data store data set including information indicative of attribute values, attribute types and attribute organization of first plurality of JSON data objects stored in a first JSON data store,
for each given JSON schema of the plurality of JSON schemas, determining a proportion value corresponding to a proportion of the first plurality of JSON objects that conform to the given JSON schema,
determining a first majority JSON schema as a JSON schema of the first plurality of JSON schemas that has the largest respectively corresponding proportion value,
for each given JSON schema of the plurality of JSON schemas, determining a first similarity score between the given JSON schema and the first majority JSON schema, with: (i) the first similarity score of the first majority JSON schema being one, and (ii) first similarity scores for each given JSON schema other than the first majority JSON schema is based upon a degree of similarity between the given JSON schema and the first majority JSON schema,
for each given JSON schema of the plurality of JSON schemas, determining a product corresponding to the given JSON schema, with the product being a product of: (i) the proportion value of the given JSON schema, multiplied by (ii) the first similarity score of the given JSON schema,
determining a first data store variance value by summing the products respectively corresponding to the JSON schema of the plurality of JSON schemas, and
selecting the first data store to supply data based, at least in part, upon the first data store variance value.

Assignees

• IBM

Inventors

• Ayyagari Phani Kumar V U
• Bhide Manish A
• Eshwar Bhavani K
• Jasti Purnachandra R

Classifications

• G06F16/80 Primary

  of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML (content-based retrieval of web data G06F16/95) · CPC title

• G06F9/45504

  Abstract machines for programme code execution, e.g. Java virtual machine [JVM], interpreters, emulators · CPC title