Semantically-relevant discovery of solutions

Patent metadata

Field	Value
Publication number	US-10133729-B2
Application number	US-201514839281-A
Country	US
Kind code	B2
Filing date	Aug 28, 2015
Priority date	Aug 28, 2015
Publication date	Nov 20, 2018
Grant date	Nov 20, 2018

Abstract

Systems, methods, and computer-readable media for providing semantically-relevant discovery of solutions are described herein. In some examples, a computing device can receive an input, such as a query. The computing device can process each word of the input sequentially to determine a semantic representation of the input. Techniques and technologies described herein determine a response to the input, such as an answer, based on the semantic representation of the input matching a semantic representation of the response. An output including one or more relevant responses to the request can then be provided to the requestor. Example techniques described herein can apply machine learning to train a model with click-through data to provide semantically-relevant discovery of solutions. Example techniques described herein can apply recurrent neural networks (RNN) and/or long short term memory (LSTM) cells in the machine learning model.

First claim

What is claimed is:

1. A system comprising:
one or more processors; and
a memory storing instructions which, when executed by one or more processors, cause the one or more processors to perform operation comprising:
receiving a query comprising one or more words;
computing a vector for individual words of the one or more words;
determining an initial hidden vector corresponding to a semantic representation of the query;
mapping, using mapping software, the one or more words of the query based on the initial hidden vector, wherein the mapping is rendered sequentially;
matching the semantic representation of the query, represented in the initial hidden vector, to a semantic representation corresponding to one or more responses;
providing, in a user interface, the one or more responses based at least in part on the matching, the user interface indicating a semantic similarity between the query and the one or more responses;
receiving click-through data associated with the one or more responses, the click-through data identifying a response from among the one or more responses as a positive match; and
training the mapping software based at least in part on the click-through data.

2. The system of claim 1 , the operations further comprising:
determining an initial hidden vector, and wherein mapping the one or more words of the query comprises:
calculating an initial vector for each word of the one or more words of the query;
applying a first parameter to a first initial vector to calculate a first vector, the first vector corresponding to a first word of the one or more words of the query;
applying a second parameter to the initial hidden vector to calculate a first vector value;
calculate a first hidden vector by combining the first vector and the first vector value; and
iteratively processing each word of the one or more words to calculate a final hidden vector value.

3. The system of claim 1 , the operations further comprising:
calculating a similarity between the semantic representations corresponding to the one or more responses and the semantic representation of the query; and
ranking the one or more responses based at least in part on the similarity.

4. The system of claim 1 , the operations further comprising:
sending the one or more responses to a user associated with the query.

5. The system of claim 1 , wherein the mapping software comprises a long short term memory cell.

6. The system of claim 1 , wherein the mapping software is trained with one or more parameters, and wherein the one or more parameters comprise one or more of:
an input weight matrix;
a recurrent weight matrix;
an input gate parameter;
an output gate parameter; and
a forget gate parameter.

7. The system of claim 1 , wherein the one or more responses comprise at least one of:
a document related to the query;
a solution to the query; or
an answer to the query.

8. A computer-implemented method comprising:
receiving a query comprising one or more words;
computing a vector for individual words of the one or more words;
determining an initial hidden vector corresponding to a semantic representation of the query;
mapping, using mapping software, the one or more words of the query based on the initial hidden vector, wherein the mapping is rendered sequentially;
matching the semantic representation of the query, represented in the initial hidden vector, to a semantic representation corresponding to one or more responses;
providing, in a user interface, the one or more responses based at least in part on the matching, the user interface indicating a semantic similarity between the query and the one or more responses;
receiving click-through data associated with the one or more responses, the click-through data identifying a response from among the one or more responses as a positive match; and
training the mapping software based at least in part on the click-through data.

9. The method of claim 8 , the sequential mapping further comprising:
applying a first parameter to a first vector to calculate a new word value, the first vector corresponding to a first word;
applying a second parameter to the initial hidden vector to calculate an initial hidden vector value;
combining the new word value and the initial hidden vector value into a hidden vector.

10. The method of claim 9 , further comprising:
iteratively calculating the new word value and the initial hidden vector value for each word of the one or more words of the query.

11. The method of claim 9 , wherein:
the first parameter is an input weight matrix; and
the second parameter is a recurrent weight matrix.

12. The method of claim 8 , wherein the mapping is performed by a long short term memory (LSTM) cell comprising one or more of:
an input gate;
an output gate; and
a forget gate.

13. The method of claim 8 , further comprising:
prior to the matching, receiving a document of a plurality of responses, the document comprising one or more words;
computing a vector for individual words of the one or more words of the document;
determining an initial hidden vector for the document;
sequentially mapping the one or more words of the document; and
determining a semantic representation of the document.

14. A non-transitory computer-readable medium storing instructions which, when executed by one or more machines, cause the one or more machines to perform operation comprising:
receiving a query comprising one or more words;
computing a vector for individual words of the one or more words;
determining an initial hidden vector corresponding to a semantic representation of the query;
sequentially mapping, using mapping software, the one or more words of the query based on the initial hidden vector;
matching the semantic representation of the query, represented in the initial hidden vector, to a semantic representation corresponding to one or more responses;
providing, in a user interface, the one or more responses based at least in part on the matching, the user interface indicating a semantic similarity between the query and the one or more responses;
receiving click-through data associated with the one or more responses, the click-through data identifying a response from among the one or more responses as a positive match; and
training the mapping software based at least in part on the click-through data.

15. The non-transitory computer-readable medium of claim 14 , the operations further comprising:
applying a first parameter to a first vector to calculate a new word value, the first vector corresponding to a first word;
applying a second parameter to the initial hidden vector to calculate an initial hidden vector value;
combining the new word value and the initial hidden vector value into a hidden vector.

16. The non-transitory computer-readable medium of claim 14 , the operations further comprising:
iteratively calculating the new word value and the initial hidden vector value for each word of the one or more words of the query.

17. The non-transitory computer-readable medium of claim 14 , the operations further comprising:
prior to the matching, receiving a document of a plurality of responses, the document comprising one or more words;
computing a vector for individual words of the one or more words of the document;
determining an initial hidden vector for the document;
sequentially mapping the one or more words of the document; and
determining a semantic representation of the document.

18. The non-transitory computer-readable med

Assignees

• Microsoft Technology Licensing Llc

Inventors

• He Xiaodong
• Gao Jianfeng
• Palangi Hamid
• Song Xinying
• Shen Yelong
• Deng Li
• Chen Jianshu

Classifications

• G06F16/3329 Primary

  Natural language query formulation · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06F16/3331

  Query processing · CPC title

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title