Systems and methods for coverage analysis of textual queries

Patent metadata

Field	Value
Publication number	US-11556572-B2
Application number	US-201916391761-A
Country	US
Kind code	B2
Filing date	Apr 23, 2019
Priority date	Apr 23, 2019
Publication date	Jan 17, 2023
Grant date	Jan 17, 2023

Abstract

A computer based system and method for assigning queries to topics and/or visualizing or analyzing query coverage may include, using a computer processor, searching, using a set of queries, over a set of text documents, to produce for each query a set of search results for the query. Each search result may include a subset of text from a text document of the set of text documents. For each query, a query vector may be calculated based on the set of search results for the query, and for each of a set of topics describing the set of text documents, a topic vector may be calculated. A report or visualization may be generated including the set of queries and the set of topics using the topic vectors and the query vectors.

First claim

What is claimed is:

1. A method for assigning queries to topics, comprising, using a computer processor:
searching, using a set of queries, over a set of text documents, to produce for each query a set of search results for the given query, each search result comprising a subset of text from a text document of the set of text documents;
for each query, calculating a query vector based on the set of search results for the given query, wherein calculating the query vector is based on an average of vectors in the set of search results;
for each of a set of topics describing the set of text documents, calculating a topic vector;
wherein each topic vector is generated by clustering phrases extracted from the set of text documents into topics and, for each topic, calculating the topic vector as the centroid of the topic; and
wherein a topic groups by proxy a group of documents among the set of text documents which are related by a common subject, the group of documents grouped based on component phrases appearing in each of the documents of the group of documents;

generating a visualization of the set of queries and the set of topics using the topic vectors and the query vectors;
determining a cosine similarity measure for the query vector and the topic vector in order to generate the visualization; and
creating a link in the visualization if the cosine similarity measure is greater than a predetermined threshold;
wherein the threshold is based on order statistics.

2. The method of claim 1 , comprising calculating the query vector by:
for each phrase in a set of phrases calculating a phrase vector for the phrase.

3. The method of claim 2 , comprising calculating the phrase vectors using a neural network.

4. The method of claim 2 , wherein each phrase vector is based on a word embedding model and measures the linguistic context of the associated phrase.

5. The method of claim 1 , wherein the text documents comprise transcripts generated by ASR (automatic speech recognition).

6. The method of claim 1 , comprising determining for each topic if the topic is covered by a query.

7. The method of claim 1 , wherein the visualization of the set of queries and the set of topics using the topic vectors and the query vectors is a combined topic-query graph using a forced directed visualization.

8. A system for assigning queries to topics, comprising, using a computer processor:
a memory and;
a processor configured to:
search, using a set of queries, over a set of text documents, to produce for each query a set of search results for the given query, each search result comprising a subset of text from a text document of the set of text documents;
for each query, calculate a query vector based on the set of search results for the given query, wherein calculating the query vector is based on an average of vectors in the set of search results;
for each of a set of topics describing the set of text documents, calculate a topic vector;
wherein each topic vector is generated by clustering phrases extracted from the set of text documents into topics and, for each topic, calculating the topic vector as the centroid of the topic; and
wherein a topic groups by proxy a group of documents among the set of text documents which are related by a common subject, the group of documents grouped based on component phrases appearing in each of the documents of the group of documents;

generate a visualization of the set of queries and the set of topics using the topic vectors and the query vectors;

determine a cosine similarity measure for the query vector and the topic vector in order to generate the visualization; and
create a link in the visualization if the cosine similarity measure is greater than a predetermined threshold;
wherein the threshold is based on order statistics.

9. The system of claim 8 , wherein the processor is configured to calculate the query vector by:
for each phrase in a set of phrases calculating a phrase vector for the phrase.

10. The system of claim 9 , wherein the processor is configured to calculate the phrase vectors using a neural network.

11. The system of claim 9 , wherein each phrase vector is based on a word embedding model and measures the linguistic context of the associated phrase.

12. The system of claim 8 , wherein the text documents comprise transcripts generated by ASR (automatic speech recognition).

13. The system of claim 8 , wherein each query vector is based on a word embedding model and measures the linguistic context of the set of search results.

14. The system of claim 8 , wherein the processor is configured to determine for each topic if the topic is covered by a query.

15. A method for analyzing queries, comprising, using a computer processor:
applying one or more queries to a set of documents, to produce query search results;
calculating a query embedding for each query, wherein calculating the query embedding is based on an average of vectors in the set of search results;
creating clusters based on the set of documents;
wherein the clusters are created by clustering phrases extracted from the set of documents into topics;

for each topic, calculating a topic vector as the centroid of the topic;
wherein a topic comprises a group of documents among the set of documents which are related by a common subject, the group of documents grouped based on component phrases appearing in each of the documents of the group of documents;

generating a visualization of the set of queries and the set of topics using the topic vectors and the query embeddings;
determining a cosine similarity measure for the query embedding and the topic vector in order to generate the visualization; and
creating a link in the visualization if the cosine similarity measure is greater than a predetermined threshold;
wherein the threshold is based on order statistics; and

determining for each cluster if the cluster is covered by a query using the topic vector and the query embeddings.

16. The method of claim 15 , comprising calculating the query embedding by:
for each phrase in a set of phrases calculating a phrase embedding for the phrase.

17. The method of claim 16 , comprising calculating the phrase embeddings using a neural network.

18. The method of claim 16 , wherein each phrase embedding is based on a word embedding model and measures the linguistic context of the associated phrase.

19. The method of claim 15 , wherein the documents comprise transcripts generated by ASR (automatic speech recognition).

20. The method of claim 15 , wherein each query embedding is based on a word embedding model and measures the linguistic context of the query search results.

Assignees

• Nice Ltd

Inventors

• Morris Robert William

Classifications

• G06F16/3328

  using graphical result space presentation or visualisation · CPC title

• G06F16/338

  Presentation of query results · CPC title

• G06F17/16

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06F16/358

  Browsing; Visualisation therefor · CPC title

• G06N20/00

  Machine learning · CPC title