Dataset engine for use within a cognitive environment

Patent metadata

Field	Value
Publication number	US-10325206-B2
Application number	US-201514629627-A
Country	US
Kind code	B2
Filing date	Feb 24, 2015
Priority date	Jun 9, 2014
Publication date	Jun 18, 2019
Grant date	Jun 18, 2019

Abstract

An apparatus for use within a cognitive information processing system environment comprising: a dataset engine, the dataset engine coupled to receive data from a plurality of data sources, the dataset engine processing the data from the plurality of data sources to establish and maintain a dynamic data ingestion and enrichment pipeline.

First claim

What is claimed is:

1. An apparatus for use within a cognitive information processing system environment, the cognitive information processing system environment comprising a plurality of data sources, a cognitive inference and learning system and a cognitive application, comprising:
a plurality of data sources, the plurality of data sources comprising a public data source and a private data source, the public data source comprising publicly available information, the private data source comprising privately managed, company specific information, the public data source comprising at least one of a social data source stored in a social data repository, public data source stored in a public data repository, a licensed data source stored in a licensed data repository and a proprietary data source stored in a proprietary data repository;
a dataset engine, the dataset engine coupled to receive data from a plurality of data sources, the dataset engine processing the data from the plurality of data sources to establish and maintain a dynamic data ingestion and enrichment pipeline, performing data enriching on the data from the plurality of data sources, and generating a sub-graph based upon the data enriching on the data from the plurality of data sources for incorporation into at least one of a cognitive graph and a private cognitive graph, the cognitive graph and the private cognitive graph each comprising a common framework for sharing data between the cognitive graph and the private cognitive graph, the cognitive graph storing publicly available information, the private cognitive graph storing privately managed, company specific information within a private cognitive platform, the private cognitive platform being contained within a private network environment, the data enriching being performed via an enrichment agent, the cognitive graph and the private cognitive graph each comprising a respective machine-readable, declarative memory system, the cognitive graph and the private cognitive graph each comprising a collection of nodes and edges that join pairs of nodes, the cognitive graph storing and learning both episodic memory and semantic memory, the sub-graph comprising a collection of nodes and edges that join pairs of nodes that are next lower than those contained in each of the cognitive graph and the private cognitive graph, the cognitive graph being stored within a repository of cognitive graphs, the cognitive inference and learning system executing on a hardware processor of an information processing system, the cognitive inference and learning system and the information processing system providing a cognitive computing function, the cognitive inference and learning system comprising a cognitive platform, the cognitive platform being coupled to and communicating with the plurality of data sources, the cognitive graph, the private cognitive graph and a cognitive application, the cognitive platform comprising a cognitive engine, the cognitive engine comprising the dataset engine, the dataset engine processing the streams data from the plurality of data sources, the dataset engine being coupled to a dataset metadata repository, the dataset metadata repository comprising curated data contained in the repository of cognitive graphs, the dataset metadata providing support of versioning of the streams of data received from the plurality of data sources, the cognitive inference and learning system using the streams of data from the dataset engine to generate cognitive insights, the cognitive insights comprising a visual data story highlighting user-specific insights, relationships and recommendations; and,
a destination, the destination receiving the cognitive insights generated using the streams of data processed by the dataset engine, the destination comprising the cognitive application, the cognitive application used by a user to interact with the cognitive insights, the cognitive application being a network cloud-based application, interaction with the cognitive insights being in the form of a response or a query, the response or query being provided to the cognitive inference and learning system, the cognitive inference and learning system providing another cognitive insight in response to the response or query from the user.

2. The apparatus of claim 1 , wherein
the dataset engine comprises a pipelines component, the pipelines component converting ingested data into streams of data for processing by the cognitive information processing system.

3. The apparatus of claim 1 , wherein
the dataset engine comprises an enrichment component, the enrichment component receiving ingested data and performing data enrichment operations on the ingested data.

4. The apparatus of claim 3 , wherein
the dataset engine comprises an enrichment agent, the enrichment agent being invoked by the enrichment component to perform the data enrichment operations on the ingested data.

5. The apparatus of claim 4 , wherein
the plurality of data sources comprise platform data; and,
the enrichment agent communicates with the platform data.

6. The apparatus of claim 5 , wherein
the platform data comprises a plurality of data sets; and,
the enrichment agent communicates with the plurality of datasets within the platform data.

7. The apparatus of claim 1 , wherein
the dataset engine comprises a storage component, the storage component caching a local version of an enriched data stream and performing incremental ingestion of the enriched data stream.

8. A non-transitory, computer-readable storage medium embodying computer program code, the computer program code comprising computer executable instructions configured for:
receiving data from a plurality of data sources, the plurality of data sources comprising a public data source and a private data source, the public data source comprising publicly available information, the private data source comprising privately managed, company specific information, the public data source comprising at least one of a social data source stored in a social data repository, public data source stored in a public data repository, a licensed data source stored in a licensed data repository and a proprietary data source stored in a proprietary data repository; and,
receiving data from the plurality of data sources to a dataset engine, the dataset engine processing the data from the plurality of data sources to establish and maintain a dynamic data ingestion and enrichment pipeline, performing data enriching on the data from the plurality of data sources, and generating a sub-graph based upon the data enriching on the data from the plurality of data sources for incorporation into at last one of a cognitive graph and a private cognitive graph, the cognitive graph and the private cognitive graph each comprising a common framework for sharing data between the cognitive graph and the private cognitive graph, the cognitive graph storing publicly available information, the private cognitive graph storing privately managed, company specific information within a private cognitive platform, the private cognitive platform being contained within a private network environment, the data enriching being performed via an enrichment agent, the cognitive graph and the private cognitive graph each comprising a respective machine-readable, declarative memory system, the cognitive graph and the private cognitive graph each comprising a collection of nodes and edges that join pairs of nodes, the cognitive graph storing and learning both episodic memory and semantic memory, the sub-graph comprising a collection of nodes and edges that join pairs of nodes that are next lower than those contained in each of the cognitive graph and the private cognitive graph, the cognitive graph being sto

Assignees

• Cognitive Scale Inc

Inventors

• Sanchez Matthew
• Ranatunga Dilum

Classifications

• G06N5/022 Primary

  Knowledge engineering; Knowledge acquisition · CPC title