Training a classification model to predict categories

Patent metadata

Field	Value
Publication number	US-10387436-B2
Application number	US-201615260105-A
Country	US
Kind code	B2
Filing date	Sep 8, 2016
Priority date	Apr 30, 2013
Publication date	Aug 20, 2019
Grant date	Aug 20, 2019

Abstract

The present invention extends to methods, systems, and computer program products for training a classification model to predict categories. In one implementation, a method identifies category mappings generated for dominant queries associated with a query log. The method identifies mappings between a first set of queries and categories shown for the first set of queries, and identifies mappings between a second set of queries and clicked products for the second set of queries. A classification model is trained based on the mappings generated for dominant queries, the mappings between queries and the shown categories, and the mappings between queries and the clicked products.

First claim

What is claimed is:

1. A computer system comprising:
one or more processors; and
system memory,
wherein:
the computer system is communicatively coupled to a query log, the query log includes query records from e-commerce queries executed against a product database, the product database comprising products, and each query record of the query records contains:
one or more categories that were used as search terms;
query results from submitting the one or more categories in a query of the product database; and
click-through information indicating the products of the product database that were selected from among the query results;

the product database uses a plurality of different categories to categorize the products of the product database;
the one or more categories are selected from among the plurality of different categories; and
the computer system performs a method of classifying the e-commerce queries that generates category mappings of dominant products of the products of the product database, the method comprising:
identifying within the query log the query records with the click-through information that indicates that one or more products of the products of the product database were selected from among corresponding query results;
qualifying the one or more products from among a plurality of products of the products of the product database, the plurality of products selected from among at least one corresponding query result returned in response to a query of a category of the plurality of different categories, the one or more products qualified by having one or more of:
a minimum number of clicks; or
a minimum number of impressions;

calculating a selection rate of the one or more products selected from among at least one corresponding query result returned in response to the query of the category of the plurality of different categories, wherein the selection rate is calculated:
based on a plurality of categories to which a product of the one or more products is assigned; and
considering an assigned primary category of the product of the one or more products;

identifying a specified top number of the products in the category of the plurality of different categories, the specified top number of the products having higher selection rates relative to other products in the category of the plurality of different categories;
calculating a category score for the category of the plurality of different categories based on product information associated with the specified top number of the products in the category of the plurality of different categories;
determining a ranking of the one or more categories based on the category score;
training a classification model with a Naïve Bayes Multinomial Model using n-grams in a bag of words associated with the e-commerce queries in the query log, based at least in part on the selection rate of the one or more products and the ranking of the one or more categories;
periodically updating the classification model based on updates to the selection rate of the one or more products and updates to the ranking of the one or more categories; and
applying the classification model to predict at least one product category of the one or more categories for a received product query.

2. The computer system of claim 1 , the method further comprising:
applying a confidence interval to the specified top number of the products of the product database that are in the category of the plurality of different categories to remove bias from category score calculations.

3. The computer system of claim 2 , the method further comprising:
varying a formula for the confidence interval.

4. The computer system of claim 1 , the method further comprising:
applying equal weighting to all of the query records, regardless of date, for calculating category scores.

5. The computer system of claim 1 , wherein identifying the specified top number of the products in the category of the plurality of different categories comprises identifying a top ten products in the category.

6. The computer system of claim 1 , wherein the calculating the selection rate comprises calculating a click-through rate based on a first number of times a shown product of the product database was shown to users and a second number of times the shown product was clicked by users.

7. The computer system of claim 1 , wherein ranking the one or more categories based on the category score comprises assigning each of the one or more categories to a category type based on the category score.

8. The computer system of claim 1 , the method further comprising:
parsing additional query record details, including, shown products of the product database shown to a user, whether or not a shown product of the shown products was added to a cart, whether or not the shown product was ordered, an order number, a primary and other category mapping for the shown product, and a position of the shown product in search results.

9. The computer system of claim 1 , the method further comprising:
modifying the click-through information to consider additional query record information comprising an add-to-cart ratio, an order ratio, and product position signals.

10. The computer system of claim 1 , wherein the one or more products is qualified by having the minimum number of clicks and the minimum number of impressions.

11. The computer system of claim 1 , wherein identifying within the query log the query records further comprises identifying the query records stored in the query log within six months.

12. A computer system for classifying e-commerce queries to generate category mappings for dominant products, the computer system comprising:
one or more processors;
system memory; and
one or more non-transitory computer storage media having stored thereon computer-executable instructions representing a query classification module, the query classification module configured to perform:
mining a query log to identify query records with click-through information that indicates that one or more products were selected through a computer network from among corresponding query results;
for each category selected from among a plurality of categories:
calculating a selection rate for each product selected from among at least one corresponding query result returned in response to a query of the category, wherein the selection rate is calculated:
based on the plurality of categories to which the each product is assigned; and
considering an assigned primary category of the each product;

identifying a specified top number of products in the category, the specified top number of the products having higher selection rates relative to other products in the category; and
calculating a category score for the category based on product information associated with the specified top number of the products in the category;

applying a confidence interval to remove bias from category score calculations;
determining a ranking of the plurality of categories based on calculated category scores;
training a classification model with a Naïve Bayes Multinomial Model using n-grams in a bag of words associated with the e-commerce queries in the query log, based at least in part on the selection rate of the one or more products and the ranking of the plurality of categories;
periodically updating the classification model based on updates to the selection rate of the one or more products and updates to the ranking of the plurality of categories; and
applying the classification model to predict at least one product category of the plurality of categories for a received product query.

Assignees

• Wal Mart Stores Inc
• Walmart Apollo Llc

Inventors

• Tholiya Namrata Pramodkumar
• Gattani Abhishek

Classifications

• G06Q30/0623 Primary

  by investigating goods or services · CPC title

• G06F16/24578 Primary

  using ranking · CPC title

• G06F16/3326

  using relevance feedback from the user, e.g. relevance feedback on documents, documents sets, document terms or passages · CPC title

• G06F16/2455

  Query execution · CPC title