Query answering

Patent metadata

Field	Value
Publication number	US-11048702-B1
Application number	US-201815890794-A
Country	US
Kind code	B1
Filing date	Feb 7, 2018
Priority date	Feb 7, 2018
Publication date	Jun 29, 2021
Grant date	Jun 29, 2021

Abstract

A method is provided. The method includes determining a number of queries for which an answer was undetermined from a knowledge database and are related to a subject. The method includes determining a period of time associated with receipt of the queries by the knowledge database. The method includes generating, based on the number of queries and the period of time, rate data indicative of a failure rate. The method includes determining that the failure rate satisfies a failure rate condition. The method includes sending text data representative of a query of the queries to a query-answering component different from the knowledge database. Other methods and systems are also provided.

First claim

What is claimed is:

1. A method, comprising:
determining, from query data representative of a first subject of first queries received by a knowledge database, that the knowledge database is lacking information to provide in response to the first subject;
using time data representative of a respective time of receipt of the first queries by the knowledge database to train a neural network to model a rate of receipt of the first queries by the knowledge database, to generate a trained neural network;
inputting a period of time to the trained neural network to generate an output of the trained neural network representative of a predicted rate of receipt of second queries by the knowledge database over the period of time, the second queries related to the first subject;
determining a measured rate of receipt of the second queries by the knowledge database over the period of time;
determining that the measured rate exceeds the predicted rate by a rate difference amount which exceeds a rate difference threshold;
in response to the determining that the measured rate exceeds the predicted rate by the rate difference amount which exceeds the rate difference threshold, sending data representative of a query of the second queries to a query-answering component different from the knowledge database;
assigning first data associated with the second queries to a first group corresponding to the first subject; and
assigning second data associated with further queries, for which an answer was undetermined from the knowledge database and which are related to a second subject, to a second group corresponding to the second subject.

2. The method of claim 1 , comprising:
at least one of:
receiving, from the query-answering component, a new entity indication that an entity associated with the first subject is to be added to the knowledge database;
generating new entity instructions to add the entity to the knowledge database; and
sending the new entity instructions to the knowledge database; or
receiving, from the query-answering component, a new relationship indication that a relationship between two existing entities of the knowledge database is to be added to the knowledge database;
generating new relationship instructions to add the relationship to the knowledge database; and
sending the new relationship instructions to the knowledge database.

3. A method, comprising:
determining a number of queries for which an answer was undetermined from a knowledge database and are related to a first subject;
determining a period of time associated with receipt of the queries by the knowledge database;
generating, based on the number of queries and the period of time, rate data indicative of a failure rate;
determining that the failure rate satisfies a failure rate condition;
in response to the determining that the failure rate satisfies the failure rate condition, sending data representative of a query of the queries to a query-answering component different from the knowledge database;
assigning first data associated with the queries to a first group corresponding to the first subject; and
assigning second data associated with further queries, for which an answer was undetermined from the knowledge database and which are related to a second subject, to a second group corresponding to the second subject.

4. The method of claim 3 , comprising:
receiving, from the query-answering component, answer data representative of at least part of an answer to the query;
based on the answer data, at least one of:
determining that an entity associated with the first subject is to be added to the knowledge database;
generating new entity instructions to add the entity to the knowledge database; and
sending the new entity instructions to the knowledge database; or
determining that a relationship between two existing entities of the knowledge database is to be added to the knowledge database;
generating new relationship instructions to add the relationship to the knowledge database; and
sending the new relationship instructions to the knowledge database.

5. The method of claim 3 , comprising:
activating a trend mode in response to the determining that the failure rate satisfies the failure rate condition;
determining that the trend mode is activated; and
sending subsequent data representative of a subsequent query related to the first subject to the query-answering component without sending the subsequent data to the knowledge database.

6. The method of claim 3 , wherein the determining the number of queries comprises determining the number of queries for which a response was at least one of: unknown, incorrect, or not received from the knowledge database within a predetermined time period.

7. The method of claim 3 , wherein at least one of: the assigning the first data or the assigning the second data uses at least one of: a clustering algorithm, locality sensitive hashing, quantized word embedding or a trained classifier.

8. The method of claim 3 , wherein the determining that the failure rate satisfies the failure rate condition comprises determining that at least one of:
the failure rate over the period of time exceeds a threshold rate;
a change in the failure rate over a predetermined period of time comprising the period of time exceeds a threshold rate change; or
the number of queries over the period of time exceeds a threshold number.

9. The method of claim 3 , wherein the number of queries is a second number of queries, the period of time is a second period of time, the rate data is second rate data, the failure rate is a second failure rate, and the method comprises:
determining a first number of queries for which an answer was undetermined from the knowledge database and are related to the first subject;
determining a first period of time associated with the first number of queries, the first period of time commencing before the second period of time; and
generating, based on the first number of queries, first rate data indicative of a first failure rate,
wherein the determining that the failure rate satisfies the failure rate condition comprises determining that an increase from the first failure rate to the second failure rate exceeds a threshold increase.

10. The method of claim 3 , comprising generating a model of a failure rate for the period of time,
wherein the determining that the failure rate satisfies the failure rate condition comprises determining that the failure rate exceeds a predicted failure rate predicted using the model by a rate difference amount which exceeds a rate difference threshold.

11. The method of claim 3 , comprising generating the data by performing speech recognition on audio data received from a user device.

12. The method of claim 3 , comprising:
receiving a data structure corresponding to the period of time and comprising identifier data for identifying the query,
wherein the generating the rate data comprises generating the rate data based on the data structure.

13. The method of claim 3 , comprising:
receiving timestamp data associated with the query, the timestamp data indicating a time at which the query was received by the knowledge database; and
generating the rate data based on the timestamp data.

14. The method of claim 3 , wherein the data is text data.

15. The method of claim 3 , comprising:
receiving an answer to the query from the query-answering component; and
providing the answer to the query in response to a plurality of queries of the queries.

16. The method of claim 3 , comprising:
identifying a representat

Assignees

• Amazon Tech Inc

Inventors

• Kumar Anjishnu
• Naslidnyk Mariia
• Duma Daniel Cristian
• Starc Janez
• Djidi Benjamin

Classifications

• G06N3/042

  Knowledge-based neural networks; Logical representations of neural networks · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N5/041

  Abduction · CPC title

• G06N5/02

  Knowledge representation; Symbolic representation · CPC title

• G06N20/00

  Machine learning · CPC title