Query rewriting and interactive inquiry framework

Patent metadata

Field	Value
Publication number	US-10654380-B2
Application number	US-201715612555-A
Country	US
Kind code	B2
Filing date	Jun 2, 2017
Priority date	Nov 18, 2016
Publication date	May 19, 2020
Grant date	May 19, 2020

Abstract

The present application describes a system and method for converting a natural language query to a standard query using a sequence-to-sequence neural network. As described herein, when a natural language query is receive, the natural language query is converted to a standard query using a sequence-to-sequence model. In some cases, the sequence-to-sequence model is associated with an attention layer. A search using the standard query is performed and various documents may be returned. The documents that result from the search are scored based, at least in part, on a determined conditional entropy of the document. The conditional entropy is determined using the natural language query and the document.

First claim

What is claimed is:

1. A method, comprising:
receiving, via a computing device, a natural language query, the natural language query being in a first format;
converting the natural language query to a machine-readable query using a sequence-to-sequence model and an associated attention layer, the machine-readable query having a second format that is different from the first format;
performing a search using the machine-readable query;
receiving a document from the search;
determining a confidence score of the document, the confidence score indicating a relevance of the document with respect to the natural language query, wherein the confidence score is based, at least in part, on a conditional entropy of the document, wherein the conditional entropy is determined using the natural language query and the document; and
based on the confidence score:
returning the document to the computing device; or
generating an additional question and providing the additional question to the computing device.

2. The method of claim 1 , further comprising returning the document only if the confidence score is above a threshold value.

3. The method of claim 1 , further comprising requesting additional input associated with the natural language query.

4. The method of claim 3 , further comprising receiving an answer associated with the additional input.

5. The method of claim 4 , further comprising:
converting the answer from a natural language format to a second machine-readable query using the sequence-to-sequence model; and
performing a search using the machine-readable query and the second standard query.

6. The method of claim 5 , further comprising returning a revised list of documents based on the machine-readable query and the second machine-readable query.

7. A system, comprising:
at least one processor; and
a memory operatively connected with the at least one processor storing computer-executable instructions that, when executed by the at least one processor, causes the at least one processor to execute a method, comprising:
receiving a natural language query, the natural language query being in a first format;
converting the natural language query to a machine-readable query using a sequence-to-sequence model, the machine-readable query having a second format that is different from the first format;
performing a search using the machine-readable query;
receiving search results from the standard query;
determining a confidence score of the search results, the confidence score indicating a relevance of the search results with respect to the natural language query, wherein the confidence score is based, at least in part, on the natural language query and the document; and
returning the document if the score of the document is greater than a threshold value.

8. The system of claim 7 , further comprising instructions for generating a question when the score of the document is below the threshold value.

9. The system of claim 8 , further comprising instructions for receiving an answer to the question.

10. The system of claim 9 , further comprising instructions for:
converting the answer from a natural language format to a second machine-readable query using the sequence-to-sequence model; and
performing a search using the machine-readable query and the second machine-readable query.

11. The system of claim 10 , further comprising instructions for returning a revised list of documents based on the machine-readable query and the second machine-readable query.

12. The system of claim 11 , further comprising scoring the revised list of documents.

13. The system of claim 7 , wherein the sequence-to-sequence model is associated with an attention layer.

14. The system of claim 13 , wherein the attention layer aggregates hidden vectors associated with the machine-readable query.

15. A method, comprising:
converting a received natural language query to a machine-readable query using a sequence-to-sequence model and an attention layer;
performing a search using the machine-readable query;
receiving a document that results from the search;
determining a confidence score of the document, the confidence score indicating a relevance of the document with respect to the natural language query, wherein the confidence score is based, at least in part, on a determined conditional entropy of the document, wherein the conditional entropy is determined, at least in part, by using the natural language query and the document; and
based on the confidence score:
returning the document; or
generating an additional question and providing the additional question to the computing device.

16. The method of claim 15 , further comprising returning the document only when the confidence score is above a threshold value.

17. The method of claim 15 , further comprising generating a question when the confidence score is below a threshold value.

18. The method of claim 17 , further comprising receiving an answer to the question.

19. The method of claim 18 , further comprising:
converting the answer from a natural language format to a second machine-readable query using the sequence-to-sequence model; and
performing a search using the machine-readable query and the second machine-readable query.

20. The method of claim 19 , further comprising returning a revised list of documents based on the machine-readable query and the second machine-readable query.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Chang Keng-Hao
• Zhang Ruofei
• Yin Zi

Classifications

• G06F16/3344

  using natural language analysis · CPC title

• G06F16/243 Primary

  Natural language query formulation · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/0445

  Physics · mapped topic

• B60N2/2806 Primary

  for securing the child seat to the vehicle · CPC title