Search Relevance Model Using Self-Adversarial Negative Sampling

Patent metadata

Field	Value
Publication number	US-2023252549-A1
Application number	US-202318107854-A
Country	US
Kind code	A1
Filing date	Feb 9, 2023
Priority date	Feb 9, 2022
Publication date	Aug 10, 2023
Grant date	—

Abstract

To train an embedding-based model to determine relevance between items and queries, an online system generates training data from previously received queries and interactions with results for the queries. The training data includes positive training examples including a query and an item with which a user performed a specific interaction after providing the query. To generate negative training examples for the query to include in the training data, the online system determines measures of similarity between items with which the specific interaction was not performed and the query. The online system may weight a loss function for the embedding-based model by the measure of similarity for a negative example, increasing the effect of a negative example including a query and an item with a larger measure of similarity. In other embodiments, the online system selects negative training examples based on the measures of similarities between items and queries in pairs.

First claim

What is claimed is:

1 . A method for generating a query-item relevance model, the method comprising:
obtaining a training dataset that comprises a plurality of training examples, wherein each of the plurality of training examples is either a positive training example or a negative training example, and wherein each of the plurality of training examples comprises:
a query comprising one or more terms, and
item data describing one or more attributes of an item;
adding a label to each of the plurality of training examples, wherein:
the label for a positive training example indicates that a specific interaction was performed with the item after the query was received, and
the label for a negative training example indicates that a specific interaction was not performed with the item after the query was received, the label further weighted by a measure of similarity between the item of the training example and the query of the training example; and
accessing a machine learning model comprising a network of a plurality of layers, the machine learning model configured to receive an input query and an input item and to generate a measure of relevance of the input item to the input query; for each training example of the plurality of training examples of the training dataset:
applying the machine learning model to the query of the training example and to the item of the training example, wherein the machine learning model outputs a measure of relevance based thereon,
generating an error term based on a difference between the measure of relevance output from the machine learning model and the label of the training example, and
backpropagating the error term to update a set of parameters of the machine learning model; and
storing the set of parameters on a non-transitory computer readable storage medium as trained parameters of the query-item relevance model.

2 . The method of claim 1 , wherein the measure of similarity between the item embedding of the item of the training example and the query embedding of the query of the training example comprises the measure of relevance between the item of the training example and the query of the training example using the stored set of parameters for the machine learning model.

3 . The method of claim 1 , wherein the measure of similarity between the item embedding of the item of the training example and the query embedding of the query of the training example comprises a cosine similarity between the item of the training example and the query of the training example.

4 . The method of claim 1 , wherein labeling each of the plurality of training examples comprises:
determining the measure of similarity between the item of the training example and the query of the training example, for each negative training example, by comparing an item embedding of the item of the training example and a query embedding of the query of the training example

5 . The method of claim 1 , wherein obtaining the training dataset further comprises:
selecting a subset of the generated one or more negative training examples for the query.

6 . The method of claim 5 , wherein selecting the subset of the generated one or more negative training examples for the query comprises:
selecting negative training examples from the generated one or more negative training examples based on a measure of similarity between the query and an item included in a negative training example.

7 . The method of claim 6 , wherein the measure of similarity between the query and the item included in the negative training example comprises the measure of relevance between the item of the negative training example and the query using the stored set of parameters for the machine learning model.

8 . The method of claim 6 , wherein the measure of similarity between the query and the item included in the negative training example comprises a cosine similarity between the item of the negative training example and the query.

9 . The method of claim 5 , wherein selecting the subset of the generated one or more negative training examples for the query comprises:
selecting negative training examples based on a probability distribution where a probability of selecting a negative training example is based on a measure of similarity between an item included in the negative training example and the query.

10 . The method of claim 1 , wherein the network of the plurality of layers comprises a query encoder configured to generate a query embedding for the query of the training example, an item encoder configured to generate an item embedding for the item of the training example, and a fusion layer configured to generate the measure of relevance from the query embedding and the item embedding.

11 . A product comprising a query-item relevance model stored on a non-transitory computer readable storage medium, wherein query-item relevance model is manufactured by a process comprising:
obtaining a training dataset that comprises a plurality of training examples, wherein each of the plurality of training examples is either a positive training example or a negative training example, and wherein each of the plurality of training examples comprises:
a query comprising one or more terms, and
item data describing one or more attributes of an item;
adding a label to each of the plurality of training examples, wherein:
the label for a positive training example indicates that a specific interaction was performed with the item after the query was received, and
the label for a negative training example indicates that a specific interaction was not performed with the item after the query was received, the label further weighted by a measure of similarity between the item of the training example and the query of the training example; and
accessing a machine learning model comprising a network of a plurality of layers, the machine learning model configured to receive an input query and an input item and to generate a measure of relevance of the input item to the input query; for each training example of the plurality of training examples of the training dataset:
applying the machine learning model to the query of the training example and to the item of the training example, wherein the machine learning model outputs a measure of relevance based thereon,
generating an error term based on a difference between the measure of relevance output from the machine learning model and the label of the training example, and
backpropagating the error term to update a set of parameters of the machine learning model; and
storing the set of parameters on a non-transitory computer readable storage medium as trained parameters of the query-item relevance model.

12 . The product of claim 11 , wherein the measure of similarity between the item embedding of the item of the training example and the query embedding of the query of the training example comprises the measure of relevance between the item of the training example and the query of the training example using the stored set of parameters for the machine learning model.

13 . The product of claim 11 , wherein the measure of similarity between the item embedding of the item of the training example and the query embedding of the query of the training example comprises a cosine similarity between the item of the training example and the query of the training example.

14 . The product of claim 11 , wherein labeling each of the plurality of training examples comprises:
determining the measure of similarity between the item of the training example and the query of the training example, for each negative

Assignees

• Maplebear Inc Dba Instacart

Inventors

• Xie Yuqing
• Na Taesik
• Manchanda Saurav

Classifications

• G06Q30/0631 Primary

  Recommending goods or services · CPC title

• G06Q30/0201 Primary

  Market modelling; Market analysis; Collecting market data · CPC title