Multiple stage filtering for natural language query processing pipelines

Patent metadata

Field	Value
Publication number	US-11726997-B2
Application number	US-202218055384-A
Country	US
Kind code	B2
Filing date	Nov 14, 2022
Priority date	Mar 31, 2021
Publication date	Aug 15, 2023
Grant date	Aug 15, 2023

Abstract

Multiple stage filtering may be implemented for natural language query processing pipelines. Natural language queries may be received at a natural language query processing system and processed through a query language processing pipeline. The query language processing pipeline may filter candidate linkages for a natural language query before performing further filtering of the candidate linkages in the natural language query processing pipeline as part of generating an intermediate representation used to execute the natural language query.

First claim

What is claimed is:

1. A system, comprising:
one or more processors; and
a memory, that stores program instructions that, when executed by the at least one processor, cause the one or more processors to implement a natural language query processing system, configured to:
receive a natural language query via an interface for the natural language query processing system that provides access to a plurality of fixed schema data sets;
process the natural language query through a query language processing pipeline that:
recognizes one or more entities in the natural language query;
determines linkages for the one or more entities in the natural language query;
detects an ambiguity for the natural language query as part of determining the linkages; and
generates an intermediate representation for the natural language query according to the determined linkages for the one or more entities; and

return, via the interface, both:
a result for the natural language query determined using the intermediate representation of the natural language query; and
an ambiguity prompt for the detected ambiguity.

2. The system of claim 1 , wherein the natural language query processing system is further configured to:
receive, via the interface, a selection corresponding to the ambiguity prompt;
update the linkages for the one or more entities in the natural language query; and
provide, via the interface, a refined result for the natural language query.

3. The system of claim 1 , wherein to detect the ambiguity for the natural language query as part of determining the linkages the query language processing pipeline evaluates respective confidence scores of the linkages with respect to a confidence threshold.

4. The system of claim 1 , wherein to determine the linkages for the one or more entities in the natural language query, the query language processing pipeline:
filters respective candidate linkages for the one or more entities in the natural language query; and
ranks the filtered candidate linkages for the one or more entities in the natural language query to determine the linkages for the one or more entities based on the ranking.

5. The system of claim 4 , wherein the ambiguity prompt provides one or more of the respective candidate linkages for the one or more entities.

6. The system of claim 5 , wherein the one or more respective candidate linkages of the ambiguity prompt have a difference with the determined one or more linkages that is less than a threshold amount.

7. The system of claim 1 , wherein the natural language query processing system is implemented as part of a service offered by a provider network.

8. A method, comprising:
receiving a natural language query via an interface for a natural language query processing system that provides access to a plurality of fixed schema data sets;
processing the natural language query through a query language processing pipeline, comprising:
recognizing one or more entities in the natural language query;
determining linkages for the one or more entities in the natural language query;
detecting an ambiguity for the natural language query as part of determining the linkages; and
generating an intermediate representation for the natural language query according to the determined linkages for the one or more entities; and

returning, via the interface, both:
a result for the natural language query determined using the intermediate representation of the natural language query; and
an ambiguity prompt for the detected ambiguity.

9. The method of claim 8 , further comprising:
receiving, via the interface, a selection corresponding to the ambiguity prompt;
updating the linkages for the one or more entities in the natural language query; and
providing, via the interface, a refined result for the natural language query.

10. The method of claim 8 , wherein detecting the ambiguity for the natural language query as part of determining the linkages comprises evaluating respective confidence scores of the linkages with respect to a confidence threshold.

11. The method of claim 8 , wherein determining the linkages for the one or more entities in the natural language query comprises:
filtering respective candidate linkages for the one or more entities in the natural language query;
ranking the filtered candidate linkages for the one or more entities in the natural language query to determine the linkages for the one or more entities based on the ranking.

12. The method of claim 11 , wherein the ambiguity prompt provides one or more of the respective candidate linkages for the one or more entities.

13. The method of claim 12 , wherein the one or more respective candidate linkages of the ambiguity prompt have a difference with the determined one or more linkages that is less than a threshold amount.

14. The method of claim 8 , wherein the natural language query processing system is implemented as part of a service offered by a provider network.

15. One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to implement:
receiving a natural language query via an interface for a natural language query processing system that provides access to a plurality of fixed schema data sets;
processing the natural language query through a query language processing pipeline, comprising:
recognizing one or more entities in the natural language query;
determining linkages for the one or more entities in the natural language query;
detecting an ambiguity for the natural language query as part of determining the linkages; and
generating an intermediate representation for the natural language query according to the determined linkages for the one or more entities; and

returning, via the interface, both:
a result for the natural language query determined using the intermediate representation of the natural language query; and
an ambiguity prompt for the detected ambiguity.

16. The one or more non-transitory computer-readable storage media of claim 15 , storing further program instructions that when executed on or across the one or more computing devices, cause the one or more computing devices to further implement:
receiving, via the interface, a selection corresponding to the ambiguity prompt;
updating the linkages for the one or more entities in the natural language query; and
providing, via the interface, a refined result for the natural language query.

17. The one or more non-transitory computer-readable storage media of claim 15 , wherein, in detecting the ambiguity for the natural language query as part of determining the linkages, the program instructions cause the one or more computing devices to implement evaluating respective confidence scores of the linkages with respect to a confidence threshold.

18. The one or more non-transitory computer-readable storage media of claim 15 , wherein, in determining the linkages for the one or more entities in the natural language query, the program instructions cause the one or more computing devices to implement:
filtering respective candidate linkages for the one or more entities in the natural language query;
ranking the filtered candidate linkages for the one or more entities in the natural language query to determine the linkages for the one or more entities based on the ranking.

19. The one or more non-transitory computer-readable storage media of claim 18 , wherein the ambiguity prompt provi

Assignees

• Amazon Tech Inc

Inventors

• Wang Jun
• Wang Zhiguo
• Revadigar Sharanabasappa Parashuram
• Nallapati Ramesh M
• Xiang Bing
• Ash Stephen Michael
• Jones Timothy
• Sengupta Sudipta
• Chakravarti Rishav
• Ng Patrick
• Jiang Jiarong
• Li Hanbo
• Weidner Donald Harold Rivers

Classifications

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F16/24522 Primary

  Translation of natural language queries to structured queries · CPC title

• G06F16/243

  Natural language query formulation · CPC title

• G06F40/295 Primary

  Named entity recognition · CPC title