Archiving of topmost ranked answers of a cognitive search

Patent metadata

Field	Value
Publication number	US-11086914-B2
Application number	US-201816154232-A
Country	US
Kind code	B2
Filing date	Oct 8, 2018
Priority date	Oct 8, 2018
Publication date	Aug 10, 2021
Grant date	Aug 10, 2021

Abstract

A method for archiving of documents of a query against a cognitive system can be provided. The cognitive system comprises at least a cognitive engine, several stored documents, and a learned model. The method comprises determining a plurality of evidence fragments, a related first list of documents and related metadata. The method also comprises removing a document from the stored documents, redetermining as second result a second list of documents, comparing the first and second list of documents, and upon determining identical documents in the compared first and second list of documents up to a confidence cliff, removing another document. Furthermore, the method comprises repeating the steps of removing, redetermining, and comparing until the first list of documents and the second list of documents differ above the confidence cliff and storing metadata of the documents of the first list, the plurality of evidence fragments, and the first query.

First claim

What is claimed is:

1. A computer-implemented method for archiving of topmost ranked documents, said method comprising:
receiving a first query into a cognitive system, wherein said cognitive system comprises a cognitive engine, a plurality of stored documents, and a related learned model;
determining a first result of said first query against said cognitive system based on said related learned model, wherein said first result comprises a plurality of evidence fragments and each evidence fragment is correlated to a document of the plurality of stored documents;
ranking said plurality of evidence fragments;
determining for said first query a first list of documents of said plurality of stored documents;
determining metadata of each document in said first list of documents;
removing a first number of documents from said plurality of stored documents, wherein said first number of documents are elements of said first list of documents, and wherein said first number of documents do not relate to topmost ranked evidence fragments;
redetermining said first result comprising determining a second list of documents of said plurality of stored documents without said first number of documents;
determining a confidence cliff;
comparing said first list of documents with said second list of documents;
determining said first list of documents and said second list of documents contain identical documents up to said confidence cliff;
in response to determining identical documents in said first list of documents and said second list of documents up to said confidence cliff, removing a second number of documents from said plurality of stored documents, wherein said second number of documents is an element of said first list of documents and said second list of documents, and wherein said second number of documents does not relate to said topmost ranked evidence fragments; and
storing said metadata of respective documents of said first list of documents, said plurality of evidence fragments, and said first query.

2. The method according to claim 1 , further comprising:
while said first list of documents and said second list of documents are equal up to said confidence cliff and respective documents in each list remain in said plurality of stored documents, repeating said step of removing a subsequent document, said step of redetermining a subsequent result, and said step of comparing said first list of documents and said second list of documents.

3. The method according to claim 1 , wherein said related learned model is agnostic to a missing document in said plurality of stored documents.

4. The method according to claim 1 , wherein said metadata comprises at least one piece of information selected from the group consisting of: a document name, a document author, a document source, a document publishing date, document bibliographic data, an International Standard Book Number, and a web-link.

5. The method according to claim 1 , further comprising storing information about said related learned model and version information of said cognitive engine used in said first query.

6. The method according to claim 1 , wherein said storing of said plurality of evidence fragments further comprises storing only said topmost evidence fragments of said plurality of evidence fragments.

7. The method according to claim 1 , wherein said confidence cliff is a predefined number of documents.

8. The method according to claim 1 , wherein said confidence cliff relates to a document in said first list of documents with a predefined confidence level.

9. The method according to claim 1 , wherein said confidence cliff is determined by:
determining a confidence level polynomial using absolute values of confidence levels of documents of said first list of documents; and
setting said confidence cliff to said document after a first local maximum of said confidence level polynomial.

10. The method according to claim 1 , wherein a list of answers relating to said first query is determined by said cognitive engine together with a scoring of said list of answers.

11. A system for archiving of topmost ranked documents comprising:
a processor; and
a computer-readable storage medium communicatively coupled to the processor and storing program instructions which, when executed by the processor, are configured to cause the processor to perform a method comprising:
receiving a first query into a cognitive system, wherein said cognitive system comprises a cognitive engine, a plurality of stored documents, and a related learned model;
determining a first result of said first query against said cognitive system based on said related learned model, wherein said first result comprises a plurality of evidence fragments and each evidence fragment is correlated to a document of the plurality of stored documents;
ranking said plurality of evidence fragments;
determining for said first query a first list of documents of said plurality of stored documents;
determining metadata of each document in said first list of documents;
removing a first number of documents from said plurality of stored documents, wherein said first number of documents are elements of said first list of documents, and wherein said first number of documents do not relate to topmost ranked evidence fragments;
redetermining said first result comprising determining a second list of documents of said plurality of stored documents without said first number of documents;
determining a confidence cliff;
comparing said first list of documents with said second list of documents;
determining said first list of documents and said second list of documents contain identical documents up to said confidence cliff;
in response to determining identical documents in said first list of documents and said second list of documents up to said confidence cliff, removing a second number of documents from said plurality of stored documents, wherein said second number of documents is an element of said first list of documents and said second list of documents, and wherein said second number of documents does not relate to said topmost ranked evidence fragments; and
storing said metadata of respective documents of said first list of documents, said plurality of evidence fragments, and said first query.

12. The system according to claim 11 , wherein said related learned model is agnostic to a missing document in said plurality of stored documents.

13. The system according to claim 11 , wherein said metadata comprises at least one piece of information selected from the group consisting of: a document name, a document author, a document source, a document publishing date, document bibliographic data, an International Standard Book Number, and a web-link.

14. The system according to claim 11 , the program instructions are further configured to cause the processor to perform a method further comprising storing information about said related learned model and version information of said cognitive engine used in said first query.

15. The system according to claim 11 , wherein the program instructions are further configured to cause the processor to perform a method further comprising storing only said topmost evidence fragments of said plurality of evidence fragments.

16. The system according to claim 11 , wherein said confidence cliff is a predefined number of documents.

17. The system according to claim 11 , wherein said confidence cliff relates to a document in said first list of documents with a predefined confidence level.

18. The system according to c

Assignees

• IBM

Inventors

• Prause Thomas
• Puig Rodriguez Ernesto
• Buendgen Reinhard T
• Mueller Joerg

Classifications

• G06F16/38

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title

• G06F16/3349 Primary

  Reuse of stored results of previous queries · CPC title

• G06F16/113

  Details of archiving (lifecycle management in storage systems G06F3/0649; point-in-time backing up or restoration of persistent data G06F11/1446) · CPC title

• G06F16/93

  Document management systems · CPC title

• G06F16/9535 Primary

  Search customisation based on user profiles and personalisation · CPC title