Neural related search query generation

Patent metadata

Field	Value
Publication number	US-11232154-B2
Application number	US-201916367849-A
Country	US
Kind code	B2
Filing date	Mar 28, 2019
Priority date	Mar 28, 2019
Publication date	Jan 25, 2022
Grant date	Jan 25, 2022

Abstract

A neural related query generation approach in a search system uses a neural encoder that reads through a source query to build a query intent vector. The approach then processes the query intent vector through a neural decoder to emit a related query. By doing so, the approach gathers information from the entire source query before generating the related query. As a result, the neural encoder-decoder approach captures long-range dependencies in the source query such as, for example, structural ordering of query keywords. The approach can be used to generate related queries for long-tail source queries, including long-tail source queries never before or not recently submitted to the search system.

First claim

The invention claimed is:

1. A method for neural related search performed by a computing system having one or more processors and storage media storing one or more programs, the one or more programs including instructions configured to perform the method and executed by the one or more processors to perform the method, the method comprising:
training a deep recurrent neural network based on a parallel corpus of keyword query pairs, each keyword query pair of the parallel corpus of keyword query pairs comprising a source keyword query and a related keyword query;
wherein the deep recurrent neural network comprises an encoder deep recurrent neural network and a decoder deep recurrent neural network;
wherein the training the deep recurrent neural network based on the parallel corpus of keyword query pairs comprises:
training the encoder deep recurrent neural network to generate query intent vectors from the source keyword queries of the parallel corpus of keyword query pairs,
initializing the decoder deep recurrent neural network with the query intent vectors,
forming input query sequences and target query sequences based on the related queries of the parallel corpus of keyword query pairs, and
training the decoder deep recurrent neural network to generate the target query sequences from the input query sequences;

based on the training the deep recurrent neural network based on the parallel corpus of keyword query pairs, inferring one or more particular related queries for a particular source query, the particular source query comprising one or more keywords, each particular related query of the one or more particular related queries comprising one or more keywords; and
causing the one or more particular related queries to be presented in a user interface at a client computing device.

2. The method of claim 1 , further comprising:
determining the parallel corpus of keyword query pairs based on one or more query logs; and
wherein at least one keyword query pair, of the parallel corpus of keyword query pairs, includes a keyword query determined to be a reformulation occurrence of another keyword query.

3. The method of claim 1 , further comprising:
determining, based on an inverted index, one or more search results for the particular source query
at least partially at a same time as performing the inferring the one or more particular related queries for the particular source query.

4. The method of claim 1 , wherein the deep recurrent neural network comprises two long short-term memory layers.

5. The method of claim 1 , further comprising:
determining one or more context limiting keywords from user profile information about a querying user; and
including the one or more context limiting keywords in both the source query and the related keyword query of a keyword query pair of the parallel corpus of keyword query pairs.

6. The method of claim 1 , wherein the training the deep recurrent neural network based on the parallel corpus of keyword query pairs comprises using an attention mechanism to force the decoder deep recurrent neural network to learn to focus on specific parts of the input query sequences when decoding the query intent vectors to generate the target query sequences.

7. The method of claim 1 , wherein the training the deep recurrent neural network based on the parallel corpus of keyword query pairs comprises obtaining word embedding vectors for keywords of source keyword queries of keyword query pairs of the parallel corpus of keyword query pairs; and wherein the training the encoder deep recurrent neural network to generate query intent vectors from source keyword queries of the parallel corpus of keyword query pairs comprises training the encoder deep recurrent neural network to generate query intent vectors from the word embedding vectors.

8. One or more non-transitory computer-readable media comprising one or more programs, the one or more programs including instructions for execution by a computing system having one or more processors, the instructions configured for:
determining a parallel corpus of keyword query pairs based on one or more query logs, each keyword query pair of the parallel corpus of keyword query pairs comprising a source keyword query and a related keyword query;
training a deep recurrent neural network based on the parallel corpus of keyword query pairs;
wherein the deep recurrent neural network comprises an encoder deep recurrent neural network and a decoder deep recurrent neural network;
wherein the instructions configured for the training the deep recurrent neural network based on the parallel corpus of keyword query pairs comprise instructions configured for:
training the encoder deep recurrent neural network to generate query intent vectors from the source keyword queries of the parallel corpus of keyword query pairs,
initializing the decoder deep recurrent neural network with the query intent vectors,
forming input query sequences and target query sequences based on the related keyword queries of the parallel corpus of keyword query pairs, and
training the decoder deep recurrent neural network to generate the target query sequences from the input query sequences;

based on the training the deep recurrent neural network based on the parallel corpus of keyword query pairs, inferring one or more particular related queries for a particular source query, the particular source query comprising one or more keywords, each related query of the one or more particular related queries comprising one or more keywords; and
causing the one or more particular related queries to be presented in a user interface at a client computing device.

9. The one or more non-transitory computer-readable media of claim 8 , the instructions for determining the parallel corpus of keyword query pairs based on the one or more query logs further comprising instructions configured for:
determining that an occurrence of a first keyword query in the one or more query logs is a reformulation of an occurrence of a second keyword query in the one or more query logs, based on determining all of the following:
the occurrence of the first keyword query is within a threshold amount of time of the occurrence of the second keyword query;
the occurrence of the first keyword query has at least one keyword in common with the occurrence of the second keyword query includes; and
the occurrence of the first keyword query and the occurrence of the second keyword query is associated with a same user.

10. The one or more non-transitory computer-readable media of claim 8 , the one or more programs further including instructions configured for:
determining, based on an inverted index, one or more search results for the particular source query
at least partially at a same time as the inferring the one or more particular related queries for the particular source query is performed.

11. The one or more non-transitory computer-readable media of claim 8 , wherein the deep recurrent neural network comprises two long short-term memory layers.

12. The one or more non-transitory computer-readable media of claim 8 , one or more programs further including instructions configured for:
determining one or more context limiting keywords from user profile information about a querying user; and
including the one or more context limiting keywords in both the source keyword query and the related keyword query of a keyword query pair of the parallel corpus of keyword query pairs.

13. The one or more non-transitory computer-readable media of claim 8 , wherein the instructions configured for training the deep recurrent neural network based on the parall

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Guo Weiwei
• Guo Lin
• Zhong Jianling
• Gao Huiji
• Long Bo

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/09

  Supervised learning · CPC title