System and method for generating a multi-task machine learning model

Patent metadata

Field	Value
Publication number	US-12488374-B2
Application number	US-202318103220-A
Country	US
Kind code	B2
Filing date	Jan 30, 2023
Priority date	Jan 30, 2023
Publication date	Dec 2, 2025
Grant date	Dec 2, 2025

Abstract

Systems and methods for generating a machine learning model to support multiple tasks for semantic retrieval, embedding and classification are disclosed. In some embodiments, a disclosed method includes: obtaining a training data set generated based on search related data and item related data associated with a website; training, based on the training data set, a machine learning model using a two-tower structure to generate an optimized set of model parameters, wherein the optimized set of model parameters minimizes a total loss function computed based on at least one of: a query classification loss, an item classification loss, and a similarity loss; and transmitting the trained machine learning model to be utilized for at least one of the following tasks: semantic item retrieval, query classification, item classification, query clustering, and item clustering.

First claim

What is claimed is:

1 . A system, comprising:
a non-transitory memory having instructions stored thereon; at least one processor operatively coupled to the non-transitory memory, and configured to read the instructions to:
obtain a training data set generated based on search related data and item related data associated with a website,
train, based on the training data set, a machine learning model having a two-tower structure to generate an optimized set of model parameters of the machine learning model, wherein a first tower of the two-tower structure is trained on the search related data and a second tower of the two-tower structure is trained on the item related data, and wherein the optimized set of model parameters minimizes a total loss function computed based on at least one of: a query classification loss, an item classification loss, and a similarity loss, and
transmit the trained machine learning model to be utilized for at least one of the following tasks: semantic item retrieval, query classification, item classification, query clustering, and item clustering.

2 . The system of claim 1 , wherein:
the training data set includes query-item-taxonomy triplets; and each of the query-item-taxonomy triplets includes: a query, an item associated with the query, and a taxonomy class associated with the query and the item.

3 . The system of claim 2 , wherein the query-item-taxonomy triplets are generated based on:
a plurality of item-taxonomy pairs each including an item and its associated taxonomy class determined based on an item page including the item on the website; user session data associated with the website; search engine marketing data of the website from paid advertisements in search results at commercial search engines; and search engine optimization data of the website from organic search results at commercial search engines.

4 . The system of claim 2 , wherein:
the two-tower structure includes a query tower and an item tower that are trained individually; the query tower learns a vector representation for each query in the query-item-taxonomy triplets to generate a query embedding during the training; the item tower learns a vector representation for each item in the query-item-taxonomy triplets to generate an item embedding during the training; the machine learning model includes a first classification layer that learns a query class for each query based on the query embedding of the query during the training; and the machine learning model includes a second classification layer that learns an item class for each item based on the item embedding of the item during the training.

5 . The system of claim 4 , wherein the total loss function is computed based on:
computing, for each query in the training data set, the query classification loss based on: the learned query class for the query, and the taxonomy class associated with the query.

6 . The system of claim 4 , wherein the total loss function is computed based on:
computing, for each item in the training data set, the item classification loss based on: the learned item class for the item, and the taxonomy class associated with the item.

7 . The system of claim 4 , wherein the total loss function is computed based on:
computing, for each query in the training data set, the similarity loss based on: a query embedding of the query, a first item embedding for a first item positively related to the query, and a second item embedding for a second item negatively related to the query.

8 . The system of claim 4 , wherein the machine learning model is trained based on:
obtaining a pre-training data set including item-taxonomy pairs and query-taxonomy pairs; and training, based on the pre-training data set, a base model using a single tower, to generate a plurality of base model parameters that minimizes a general classification loss, wherein the plurality of base model parameters are used as initial parameters for both the query tower and the item tower in the machine learning model when training the machine learning model.

9 . The system of claim 8 , wherein:
each item-taxonomy pair includes an item and its paired taxonomy class; each query-taxonomy pair includes a query and its paired taxonomy class; the single tower learns a vector representation for each item in the item-taxonomy pairs and each query in the query-taxonomy pairs to generate a corresponding general embedding; the base model includes a classification layer that learns a general classification class for each query and each item based on the corresponding general embedding; and the general classification loss is computed, for each query and each item in the pre-training data set, based on: the general classification class, and the paired taxonomy class.

10 . The system of claim 4 , wherein in response to a search query submitted after the machine learning model is trained:
the query tower is configured to infer a vector representation for the search query to generate a new query embedding; the machine learning model is configured to determine one or more items whose item embeddings are within a predetermined pair-wise distance to the new query embedding in an embedding space; and the one or more items are identified for semantic retrieval in response to the search query.

11 . A computer-implemented method, comprising:
obtaining a training data set generated based on search related data and item related data associated with a website; training, based on the training data set, a machine learning model having a two-tower structure to generate an optimized set of model parameters of the machine learning model, wherein a first tower of the two-tower structure is trained on the search related data and a second tower of the two-tower structure is trained on the item related data, and wherein the optimized set of model parameters minimizes a total loss function computed based on at least one of: a query classification loss, an item classification loss, and a similarity loss; and transmitting the trained machine learning model to be utilized for at least one of the following tasks: semantic item retrieval, query classification, item classification, query clustering, and item clustering.

12 . The computer-implemented method of claim 11 , wherein:
the training data set includes query-item-taxonomy triplets; and each of the query-item-taxonomy triplets includes: a query, an item associated with the query, and a taxonomy class associated with the query and the item.

13 . The computer-implemented method of claim 12 , wherein the query-item-taxonomy triplets are generated based on:
a plurality of item-taxonomy pairs each including an item and its associated taxonomy class determined based on an item page including the item on the website; user session data associated with the website; search engine marketing data of the website from paid advertisements in search results at commercial search engines; and
search engine optimization data of the website from organic search results at commercial search engines.

14 . The computer-implemented method of claim 12 , wherein:
the two-tower structure includes a query tower and an item tower that are trained individually; the query tower learns a vector representation for each query in the query-item-taxonomy triplets to generate a query embedding during the training; the item tower learns a vector representation for each item in the query-item-taxonomy triplets to generate an item embedding during the training; the machine learning model includes a first classification layer that learns

Assignees

• Walmart Apollo Llc

Inventors

• Wang Zigeng
• Zhai Bohan

Classifications

• G06Q30/0625

  by formulating product or service queries, e.g. using keywords or predefined options · CPC title

• G06Q30/0255 Primary

  based on user history · CPC title

• G06Q30/0202

  Market predictions or forecasting for commercial activities · CPC title

• G06Q30/0631 Primary

  Recommending goods or services · CPC title