User click modelling in search queries

Patent metadata

Field	Value
Publication number	US-12292895-B2
Application number	US-202318529025-A
Country	US
Kind code	B2
Filing date	Dec 5, 2023
Priority date	Oct 30, 2020
Publication date	May 6, 2025
Grant date	May 6, 2025

Abstract

A method for ranking documents in search results includes defining a first training data set, the first training data set including, for each of a plurality of user queries, information respective of a document selected by a user from results responsive to the query and information respective of one or more documents within an observation window after the selected document in the results, and defining a second training data set, the second training data set including, for each of the plurality of user queries, information respective of the selected document. The method further includes training a first machine learning model with the first training data set, training a second machine learning model with the second training data set, and ranking documents of a further search result set according to the output of the first machine learning model and the output of the second machine learning model.

First claim

What is claimed is:

1. A method for ranking documents in search results, the method comprising:
retrieving a plurality of search result sets, each search result set being associated with a user query and comprising an ordered plurality of documents;
defining a training data set based on the plurality of search result sets by, for each search result set:
determining an observation window including a pre-defined number of documents ordered after a responsive document from the search result set, the responsive document representative of a document selected by a user from the search result set; and
discarding documents from the search result set that are ordered below the pre-defined number of documents after the responsive document;

training a machine learning model via the training data set;
receiving a further user query;
presenting a list of responsive documents ranked by the trained machine learning model;
receiving indication of a responsive document from the presented list of responsive documents;
processing the list of responsive documents to discard documents from the list of responsive documents that are outside of the observation window from the indicated responsive document; and
adding the processed list of responsive documents to the training data set,
wherein, by not including the discarded documents, the training data set reduces a bias representative of a user selection of the responsive document over the discarded documents at the machine learning model.

2. The method of claim 1 , wherein training the machine learning model comprises:
conducting reinforcement learning on the machine learning model to maximize a prediction accuracy of the machine learning model.

3. The method of claim 1 , wherein the observation window includes between one and three documents after the responsive document.

4. The method of claim 1 , wherein:
the plurality of search result sets comprises:
a plurality of previous search result sets; and
one or more current search result sets; and

training the machine learning model with the defined training data set comprises:
batch training the machine learning model according to the previous search result sets; and
conducting reinforcement learning on the machine learning model according to the one or more current search result sets.

5. The method of claim 4 , wherein conducting reinforcement learning comprises maximizing a prediction accuracy of the machine learning model.

6. The method of claim 1 , wherein presenting the list of responsive documents comprises displaying the ranked further search result set on a user device.

7. A system comprising:
a non-transitory, computer-readable medium storing instructions; and
a processor configured to execute the instructions to:
retrieve a plurality of search result sets, each search result set being associated with a user query and comprising an ordered plurality of documents;
define a training data set based on the plurality of search result sets by, for each search result set:
determine an observation window including a pre-defined number of documents ordered after a responsive document from the search result set, the responsive document representative of a document selected by a user from the search result set; and
discard documents from the search result set that are ordered below the pre-defined number of documents after the responsive document;

train a machine learning model via the training data set;
receive a further user query;
present a list of responsive documents ranked by the trained machine learning model;
receiving indication of a responsive document from the presented list of responsive documents;
processing the list of responsive documents to discard documents from the list of responsive documents that are outside of the observation window from the indicated responsive document; and
adding the processed list of responsive documents to the training data set,
wherein, by not including the discarded documents, the training data set reduces a bias representative of a user selection of the responsive document over the discarded documents at the machine learning model.

8. The system of claim 7 , wherein training the machine learning model comprises:
conducting reinforcement learning on the machine learning model to maximize a prediction accuracy of the machine learning model.

9. The system of claim 7 , wherein the observation window includes between one and three documents after the responsive document.

10. The system of claim 7 , wherein:
the plurality of search result sets comprises:
a plurality of previous search result sets; and
one or more current search result sets; and

training the machine learning model with the defined training data set comprises:
batch training the machine learning model according to the previous search result sets; and
conducting reinforcement learning on the machine learning model according to the one or more current search result sets.

11. The system of claim 10 , wherein conducting reinforcement learning comprises maximizing a prediction accuracy of the machine learning model.

12. The system of claim 7 , wherein presenting the list of responsive documents comprises displaying the ranked further search result set on a user device.

13. A method for presenting search results, the method comprising:
retrieving a plurality of search result sets, each search result set being associated with a user query and comprising an ordered plurality of documents;
defining a first training data set based on the plurality of search result sets by, for each search result set:
determining an observation window including a pre-defined number of documents ordered after a responsive document from the search result set, the responsive document representative of a document selected by a user from the search result set; and
discarding documents from the search result set that are ordered below the pre-defined number of documents after the responsive document;

training a first machine learning model via the first training data set;
defining a second training data set based on the responsive document from each of the plurality of search result sets;
training a second machine learning model via the second training data set;
receiving a further user query;
presenting a list of responsive documents ranked according to the first and second trained machine learning models;
receiving indication of a responsive document from the presented list of responsive documents;
processing the list of responsive documents to discard documents from the list of responsive documents that are outside of the observation window from the indicated responsive document; and
adding the processed list of responsive documents to the training data set,
wherein, by not including the discarded documents, the first training data set reduces a bias representative of a user selection of the responsive document over the discarded documents at the first machine learning model.

14. The method of claim 13 , wherein training the first machine learning model and training the second machine learning model comprises:
conducting reinforcement learning on the first machine learning model and the second machine learning model to maximize a reward, the reward comprising a combination of a prediction accuracy of the first machine learning model and a prediction accuracy of the second machine learning model.

15. The method of claim 13 , wherein the observation window includes between one and three documents after the responsive document.

16. The method of claim 13 , wherein:

Assignees

• Home Depot Product Authority Llc

Inventors

• Zhou Jianghong
• Zahiri Sayyed
• Hughes Simon
• Kallumadi Surya
• Al Jadda Khalifeh
• Agichtein Eugene

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06F16/93

  Document management systems · CPC title

• G06F16/248

  Presentation of query results · CPC title