Grading Sources and Managing Evidence for Intelligence Analysis

Patent metadata

Field	Value
Publication number	US-2016140446-A1
Application number	US-201414547246-A
Country	US
Kind code	A1
Filing date	Nov 19, 2014
Priority date	Nov 19, 2014
Publication date	May 19, 2016
Grant date	—

Abstract

Mechanisms for evaluating an evidential statement in a corpus of evidence are provided. An evidential statement is received for determining a level of confidence in a hypothetical ontological link of an ontology. A source of the evidential statement is identified and a grading of the source of the evidential statement is determined based on a source grading measurement value indicative of a degree of reliability and credibility of the source. An indication of trustworthiness of the evidential statement is generated based on the source grading measurement value. A representation of the indication of trustworthiness of the evidential statement is output in association with the evidential statement.

First claim

What is claimed is:

1 . A method, in a data processing system comprising a processor and a memory, for evaluating an evidential statement in a corpus of evidence, comprising:
receiving, by the data processing system, an evidential statement for determining a level of confidence in a hypothetical ontological link of an ontology; identifying, by the data processing system, a source of the evidential statement; determining, by the data processing system, a grading of the source of the evidential statement based on a source grading measurement value indicative of a degree of reliability and credibility of the source; generating, by the data processing system, an indication of trustworthiness of the evidential statement based on the source grading measurement value; and outputting, by the data processing system, a representation of the indication of trustworthiness of the evidential statement in association with the evidential statement.

2 . The method of claim 1 , wherein grading the source of the evidential statement comprises generating a weight value to be applied to confidence scores calculated for answers to questions generated based on evidential statements in the corpus associated with the source.

3 . The method of claim 1 , further comprising managing a corpus of evidence data based on the grading of the source of the evidential statement, wherein managing the corpus comprises one of maintaining an amount of evidence data from the source in the corpus, increasing the amount of evidence data from the source that is in the corpus, or decreasing the amount of evidence data from the source in the corpus.

4 . The method of claim 3 , wherein managing the corpus of evidence data comprises comparing the source grading measurement value to one or more predetermined threshold values and performing a corpus management operation based on results of the comparison.

5 . The method of claim 4 , wherein, in response to the comparison indicating that the source grading measurement value is above a first threshold, ingesting addition evidence data from the source into the corpus.

6 . The method of claim 4 , wherein, in response to the comparison indicating that the source grading measurement value is above a first threshold, sending a request to a user specifying the source and the source grading measurement value and a recommendation to obtain additional evidence data from the source.

7 . The method of claim 4 , wherein, in response to the comparison indicating that the source grading measurement is below a second threshold, removing evidence data associated with the source from the corpus.

8 . The method of claim 3 , wherein the corpus is managed to temporarily modify the corpus being used to evaluate candidate answers to a question set used to evaluate a hypothetical ontological link of an ontology.

9 . The method of claim 1 , wherein determining a grading of the source comprises evaluating current and historical characteristics of the source, wherein the historical characteristics of the source comprises information indicative of results from previous utilizations of evidence data from the source when evaluating answers to previously processed questions.

10 . The method of claim 9 , wherein the current and historical characteristics are maintained in a source profile data structure of a source directory associated with the data processing system, and wherein the source profile data structure stores data indicating a number of previous uses of evidence data from the source and a number of previous positive uses of the evidence data from the source.

11 . A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to:
receive an evidential statement for determining a level of confidence in a hypothetical ontological link of an ontology; identify a source of the evidential statement; determine a grading of the source of the evidential statement based on a source grading measurement value indicative of a degree of reliability and credibility of the source; generate an indication of trustworthiness of the evidential statement based on the source grading measurement value; and output a representation of the indication of trustworthiness of the evidential statement in association with the evidential statement.

12 . The computer program product of claim 11 , wherein grading the source of the evidential statement comprises generating a weight value to be applied to confidence scores calculated for answers to questions generated based on evidential statements in the corpus associated with the source.

13 . The computer program product of claim 11 , further comprising managing a corpus of evidence data based on the grading of the source of the evidential statement, wherein managing the corpus comprises one of maintaining an amount of evidence data from the source in the corpus, increasing the amount of evidence data from the source that is in the corpus, or decreasing the amount of evidence data from the source in the corpus.

14 . The computer program product of claim 13 , wherein managing the corpus of evidence data comprises comparing the source grading measurement value to one or more predetermined threshold values and performing a corpus management operation based on results of the comparison.

15 . The computer program product of claim 14 , wherein, in response to the comparison indicating that the source grading measurement value is above a first threshold, ingesting addition evidence data from the source into the corpus.

16 . The computer program product of claim 14 , wherein, in response to the comparison indicating that the source grading measurement value is above a first threshold, sending a request to a user specifying the source and the source grading measurement value and a recommendation to obtain additional evidence data from the source.

17 . The computer program product of claim 14 , wherein, in response to the comparison indicating that the source grading measurement is below a second threshold, removing evidence data associated with the source from the corpus.

18 . The computer program product of claim 13 , wherein the corpus is managed to temporarily modify the corpus being used to evaluate candidate answers to a question set used to evaluate a hypothetical ontological link of an ontology.

19 . The computer program product of claim 11 , wherein determining a grading of the source comprises evaluating current and historical characteristics of the source, wherein the historical characteristics of the source comprises information indicative of results from previous utilizations of evidence data from the source when evaluating answers to previously processed questions, and wherein the current and historical characteristics are maintained in a source profile data structure of a source directory associated with the data processing system, and wherein the source profile data structure stores data indicating a number of previous uses of evidence data from the source and a number of previous positive uses of the evidence data from the source.

20 . An apparatus comprising:
a processor; and a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to: receive an evidential statement for determining a level of confidence in a hypothetical ont

Assignees

• IBM

Inventors

• Adderly Darryl M
• Allen Corville O
• Tucker Robert K

Classifications

• G06N5/048 Primary

  Fuzzy inferencing · CPC title

• G06N5/022 Primary

  Knowledge engineering; Knowledge acquisition · CPC title