Verification of natural language processing derived attributes

Patent metadata

Field	Value
Publication number	US-2016188535-A1
Application number	US-201414584414-A
Country	US
Kind code	A1
Filing date	Dec 29, 2014
Priority date	Dec 29, 2014
Publication date	Jun 30, 2016
Grant date	—

Abstract

System, method, and computer program product to identify candidate values to provide to a deep question answering (QA) system as part of a case, by receiving a case, wherein the case includes a plurality of documents for evaluation by the deep QA system, evaluating the plurality of documents using natural language processing (NLP) to identify one or more concepts reflected by text content within the plurality of documents in the case, wherein the plurality of documents includes a plurality of distinct values for at least a first one of the concepts, selecting, from the plurality of distinct values, a candidate value for the first concept to provide to the deep QA system to process the case, and prior to submitting the case to the deep QA system, returning at least the candidate value selected for the first concept to present in a user interface.

First claim

What is claimed is:

1 . A computer implemented method for identifying candidate values to provide to a deep question answering (QA) system as part of a case, the method comprising:
receiving a case, wherein the case includes a plurality of documents for evaluation by the deep QA system; evaluating the plurality of documents using natural language processing (NLP) to identify one or more concepts reflected by text content within the plurality of documents in the case, wherein the plurality of documents includes a plurality of distinct values for at least a first one of the concepts; selecting, from the plurality of distinct values, a candidate value for the first concept to provide to the deep QA system to process the case; and prior to submitting the case to the deep QA system, returning at least the candidate value selected for the first concept to present in a user interface.

2 . The method of claim 1 , further comprising submitting the case and an indication of the candidate value to the deep QA system for processing.

3 . The method of claim 1 , wherein the decision comprises a treatment recommendation, and wherein the case is an electronic medical record (EMR).

4 . The method of claim 1 , wherein one or more of the plurality of distinct values includes a respective collection of one or more attributes, and wherein the method further comprises building a model for selecting the concept values and attributes from a plurality of training cases, each training case providing a plurality of documents and an indication of a preferred value for each concept which can have multiple values.

5 . The method of claim 1 , further comprising, receiving a request to view other values from the plurality of distinct values in addition to the candidate value.

6 . The method of claim 5 , further comprising receiving a selection of one of the plurality of distinct values to submit with the case as an indication of the candidate value to the deep QA system.

7 . The method of claim 1 , wherein the selection is related to a confidence score derived from the model for each of the plurality of distinct values.

8 . A system for identifying candidate values to provide to a deep question answering (QA) system as part of a case, the system comprising:
one or more computer processors; a memory containing a program which when executed by the one or more computer processors performs an operation, the operation comprising:
receiving a case, wherein the case includes a plurality of documents for evaluation by the deep QA system;
evaluating the plurality of documents using natural language processing (NLP) to identify one or more concepts reflected by text content within the plurality of documents in the case, wherein the plurality of documents includes a plurality of distinct values for at least a first one of the concepts;
selecting, from the plurality of distinct values, a candidate value for the first concept to provide to the deep QA system to process the case; and
prior to submitting the case to the deep QA system, returning at least the candidate value selected for the first concept to present in a user interface.

9 . The system of claim 8 , further comprising submitting the case and an indication of the candidate value to the deep QA system for processing.

10 . The system of claim 8 , wherein the decision comprises a treatment recommendation, and wherein the case is an electronic medical record (EMR).

11 . The system of claim 8 , wherein one or more of the plurality of distinct values includes a respective collection of one or more attributes, and wherein the system further comprises building a model for selecting the concept values and attributes from a plurality of training cases, each training case providing a plurality of documents and an indication of a preferred value for each concept which can have multiple values.

12 . The system of claim 8 , further comprising, receiving a request to view other values from the plurality of distinct values in addition to the candidate value.

13 . The system of claim 12 , further comprising receiving a selection of one of the plurality of distinct values to submit with the case as an indication of the candidate value to the deep QA system.

14 . The system of claim 8 , wherein the selection is related to a confidence score derived from the model for each of the plurality of distinct values.

15 . A computer program product, comprising:
a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code for identifying candidate values to provide to a deep question answering (QA) system as part of a case comprising: computer-readable program code configured to receive a case, wherein the case includes a plurality of documents for evaluation by the deep QA system; computer-readable program code configured to evaluate the plurality of documents using natural language processing (NLP) to identify one or more concepts reflected by text content within the plurality of documents in the case, wherein the plurality of documents includes a plurality of distinct values for at least a first one of the concepts; computer-readable program code configured to select, from the plurality of distinct values, a candidate value for the first concept to provide to the deep QA system to process the case; and computer-readable program code configured to, prior to submitting the case to the deep QA system, return at least the candidate value selected for the first concept to present in a user interface.

16 . The computer-program product of claim 15 , further comprising computer-readable program code configured to submit the case and an indication of the candidate value to the deep QA system for processing.

17 . The computer-program product of claim 15 , wherein the decision comprises a treatment recommendation, and wherein the case is an electronic medical record (EMR).

18 . The computer-program product of claim 15 , wherein one or more of the plurality of distinct values includes a respective collection of one or more attributes, and wherein the computer program product further comprises computer-readable program code configured to build a model for selecting the concept values and attributes from a plurality of training cases, each training case providing a plurality of documents and an indication of a preferred value for each concept which can have multiple values.

19 . The computer-program product of claim 15 , further comprising, computer-readable program code configured to receive a request to view other values from the plurality of distinct values in addition to the candidate value.

20 . The computer-program product of claim 19 , further comprising computer-readable program code configured to receive a selection of one of the plurality of distinct values to submit with the case as an indication of the candidate value to the deep QA system.

21 . The computer-program product of claim 15 , wherein the selection is related to a confidence score derived from the model for each of the plurality of distinct values.

Assignees

• IBM

Inventors

• Allen Corville O
• Delima Roberto
• Eggebraaten Thomas J
• Petri John E
• Setnes Marie L
• Wildt Patrick M

Classifications

• G06F40/30

  Semantic analysis · CPC title

• G06F16/93 Primary

  Document management systems · CPC title

• G16H10/60

  for patient-specific data, e.g. for electronic patient records · CPC title

• G06F17/20 Primary

  Physics · mapped topic

• G06F19/322

  Physics · mapped topic