Question answering with entailment analysis

Patent metadata

Field	Value
Publication number	US-10783159-B2
Application number	US-201414575462-A
Country	US
Kind code	B2
Filing date	Dec 18, 2014
Priority date	Dec 18, 2014
Publication date	Sep 22, 2020
Grant date	Sep 22, 2020

Abstract

Techniques for question answering involve receiving, from a user, a text input expressing a question in natural language. In response to the question, a text output expressing an answer to the question may be generated. A plurality of documents comprising natural language text may be analyzed, involving mapping the question to one or more hypotheses, analyzing at least one passage of text in at least one of the documents to determine whether the passage entails at least one of the hypotheses, and in response to determining that the passage entails at least one of the hypotheses, identifying the passage as providing supporting evidence for the answer to the question. The answer and the at least one passage identified as providing supporting evidence for the answer may be presented to the user in response to the text input.

First claim

What is claimed is:

1. A method comprising:
receiving, from a user, a text input expressing a question in natural language;
generating, in response to the question, a text output expressing an answer to the question;
analyzing, using a language analysis component implemented via at least one processor, a plurality of documents comprising natural language text, the analyzing comprising:
converting the question to one or more hypotheses, wherein the one or more hypotheses identify one or more constraints on answers that will satisfy the question;
analyzing at least one passage of text in at least one of the plurality of documents to extract one or more assertions made by the at least one passage;
identifying one or more semantic relations shared between the one or more assertions made by the at least one passage and the one or more hypotheses;
determining, based on the identified one or more semantic relations, that truth of at least one hypothesis of the one or more hypotheses logically follows from truth of at least one of the one or more assertions extracted from the at least one passage of text; and
in response to determining that the at least one passage entails the at least one hypothesis, identifying the at least one passage as providing supporting evidence for the answer to the question; and

presenting to the user, in response to the text input, the answer and the at least one passage from the at least one of the documents identified as providing supporting evidence for the answer.

2. The method of claim 1 , wherein the analyzing further comprises:
scoring passages in at least some of the plurality of documents based at least in part on strength of the passages' supporting evidence for the answer; and
selecting one or more of the passages for presentation to the user based on the passages' scores.

3. The method of claim 1 , wherein generating the text output expressing the answer comprises determining the answer based at least in part on the text in the at least one passage.

4. The method of claim 3 , wherein determining the answer comprises selecting between at least a first answer supported by a first set of one or more passages of text in the plurality of documents and a different second answer supported by a second set of one or more passages of text in the plurality of documents.

5. The method of claim 4 , wherein selecting between at least the first answer and the second answer comprises comparing the first and second sets of passages based at least in part on (a) how many documents are represented in the first set of passages versus how many documents are represented in the second set of passages, and/or (b) strength of the first set of passages' supporting evidence for the first answer versus strength of the second set of passages' supporting evidence for the second answer.

6. The method of claim 3 , wherein determining the answer comprises:
identifying a plurality of constraints posed by the question;
matching different constraints of the plurality of constraints to passages in different documents of the plurality of documents; and
identifying an answer supported by the passages in combination matched to the plurality of constraints of the question.

7. The method of claim 6 , further comprising selecting one or more of the matched passages for presentation to the user based at least in part on how many of the plurality of constraints of the question are matched to the one or more matched passages.

8. The method of claim 1 , wherein determining that truth of the at least one hypothesis logically follows from truth of the at least one assertion comprises determining that the at least one passage entails the at least one hypothesis.

9. At least one computer-readable storage medium storing computer-executable instructions that, when executed, perform a method comprising:
receiving, from a user, a text input expressing a question in natural language;
generating, in response to the question, a text output expressing an answer to the question;
analyzing a plurality of documents comprising natural language text, the analyzing comprising:
converting the question to one or more hypotheses, wherein the one or more hypotheses identify one or more constraints on answers that will satisfy the question;
analyzing at least one passage of text in at least one of the plurality of documents to extract one or more assertions made by the at least one passage;
identifying one or more semantic relations shared between the one or more assertions made by the at least one passage and the one or more hypotheses;
determining, based on the identified one or more semantic relations, that truth of at least one hypothesis of the one or more hypotheses logically follows from truth of at least one of the one or more assertions extracted from the at least one passage of text; and
in response to determining that the at least one passage entails the at least one hypothesis, identifying the at least one passage as providing supporting evidence for the answer to the question; and

presenting to the user, in response to the text input, the answer and the at least one passage from the at least one of the documents identified as providing supporting evidence for the answer.

10. The at least one computer-readable storage medium of claim 9 , wherein the analyzing further comprises:
scoring passages in at least some of the plurality of documents based at least in part on strength of the passages' supporting evidence for the answer; and
selecting one or more of the passages for presentation to the user based on the passages' scores.

11. The at least one computer-readable storage medium of claim 9 , wherein generating the text output expressing the answer comprises determining the answer based at least in part on the text in the at least one passage.

12. The at least one computer-readable storage medium of claim 11 , wherein determining the answer comprises selecting between at least a first answer supported by a first set of one or more passages of text in the plurality of documents and a different second answer supported by a second set of one or more passages of text in the plurality of documents.

13. The at least one computer-readable storage medium of claim 12 , wherein selecting between at least the first answer and the second answer comprises comparing the first and second sets of passages based at least in part on (a) how many documents are represented in the first set of passages versus how many documents are represented in the second set of passages, and/or (b) strength of the first set of passages' supporting evidence for the first answer versus strength of the second set of passages' supporting evidence for the second answer.

14. The at least one computer-readable storage medium of claim 11 , wherein determining the answer comprises:
identifying a plurality of constraints posed by the question;
matching different constraints of the plurality of constraints to passages in different documents of the plurality of documents; and
identifying an answer supported by the passages in combination matched to the plurality of constraints of the question.

15. The at least one computer-readable storage medium of claim 14 , wherein the method further comprises selecting one or more of the matched passages for presentation to the user based at least in part on how many of the plurality of constraints of the question are matched to the one or more matched passages.

16. The at least one computer-readable storage medium of claim 9 , wherein determining that truth of the at least one hypothesis logically

Assignees

• Nuance Communications Inc

Inventors

• Boston Marisa Ferrara
• Crouch Richard Stamford
• Ozcan Ali Erdem
• Stubley Peter

Classifications

• G06F16/24578

  using ranking · CPC title

• G06F16/248 Primary

  Presentation of query results · CPC title