Generating ground truth for questions based on data found in structured resources

Patent metadata

Field	Value
Publication number	US-10482180-B2
Application number	US-201715816089-A
Country	US
Kind code	B2
Filing date	Nov 17, 2017
Priority date	Nov 17, 2017
Publication date	Nov 19, 2019
Grant date	Nov 19, 2019

Abstract

Ground truth for a cognitive system is generated from a structured resource such as a table by identifying a subject of the structured resource and field headers. Linguistic analysis is performed on a given header to establish an interrogative context, and a question is generated relating to the subject based on the interrogative context, including an implementation of one or more mathematical operators. The question is generated using a question template, and has a question phrase based on the interrogative context, an operator phrase based on the selected operator, and a keyword phrase based on the subject. An answer to the question is determined by carrying out a computation that applies the selected operator(s) to one or more of the data values, to form a question-and-answer pair that is added to the ground truth. A filtering step is preferably used to ensure that the question-and-answer pair is valid.

First claim

What is claimed is:

1. A method of providing ground truth for a cognitive system comprising:
receiving a structured resource having a set of data values, by executing first instructions in a computer system;
receiving a set of operators, by executing second instructions in the computer system;
identifying a subject of the structured resource and at least one field header of the structured resource, by executing third instructions in the computer system;
performing linguistic analysis on the field header to determine an interrogative context, by executing fourth instructions in the computer system;
generating at least one question relating to the subject based on the interrogative context wherein the question includes an implementation of a selected one of the operators, by executing fifth instructions in the computer system; and
determining an answer for the question to form a question-and-answer pair.

2. The method of claim 1 wherein the question includes a question phrase based on the interrogative context, an operator phrase based on the selected operator, and a keyword phrase based on the subject.

3. The method of claim 1 wherein the data values are numerical values and the operators are mathematical operators.

4. The method of claim 1 wherein the question is generated using a question template.

5. The method of claim 1 wherein said determining the answer includes carrying out a computation by applying the selected operator to one or more of the data values.

6. The method of claim 1 further comprising determining that the question-and-answer pair is valid.

7. The method of claim 1 further comprising:
storing the question-and-answer pair as part of the ground truth for the cognitive system; and
using the cognitive system to formulate a response to a natural language query.

8. A computer system comprising:
one or more processors which process program instructions;
a memory device connected to said one or more processors; and
program instructions residing in said memory device for providing ground truth to a cognitive system by receiving a structured resource having a set of data values, receiving a set of operators, identifying a subject of the structured resource and at least one field header of the structured resource, performing linguistic analysis on the field header to determine an interrogative context, generating at least one question relating to the subject based on the interrogative context wherein the question includes an implementation of a selected one of the operators, and determining an answer for the question to form a question-and-answer pair.

9. The computer system of claim 8 wherein the question includes a question phrase based on the interrogative context, an operator phrase based on the selected operator, and a keyword phrase based on the subject.

10. The computer system of claim 8 wherein the data values are numerical values and the operators are mathematical operators.

11. The computer system of claim 8 wherein the question is generated using a question template.

12. The computer system of claim 8 wherein determining the answer includes carrying out a computation by applying the selected operator to one or more of the data values.

13. The computer system of claim 8 wherein said program instructions further determine that the question-and-answer pair is valid.

14. The computer system of claim 8 wherein said program instructions further store the question-and-answer pair as part of the ground truth for the cognitive system.

15. A computer program product comprising:
a computer readable storage medium; and
program instructions residing in said storage medium for providing ground truth to a cognitive system by receiving a structured resource having a set of data values, receiving a set of operators, identifying a subject of the structured resource and at least one field header of the structured resource, performing linguistic analysis on the field header to determine an interrogative context, generating at least one question relating to the subject based on the interrogative context wherein the question includes an implementation of a selected one of the operators, and determining an answer for the question to form a question-and-answer pair.

16. The computer program product of claim 15 wherein the question includes a question phrase based on the interrogative context, an operator phrase based on the selected operator, and a keyword phrase based on the subject.

17. The computer program product of claim 15 wherein the data values are numerical values and the operators are mathematical operators.

18. The computer program product of claim 15 wherein the question is generated using a question template.

19. The computer program product of claim 15 wherein determining the answer includes carrying out a computation by applying the selected operator to one or more of the data values.

20. The computer program product of claim 15 wherein said program instructions further determine that the question-and-answer pair is valid.

Assignees

• IBM

Inventors

• Santos Moraes Priscilla
• Banks Kathryn V
• Tecuci Dan G

Classifications

• G06F16/3329 Primary

  Natural language query formulation · CPC title

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06F40/279

  Recognition of textual entities · CPC title

• G06F40/186

  Templates · CPC title

• G06F16/3344

  using natural language analysis · CPC title