System and method of tuning item classification

Patent metadata

Field	Value
Publication number	US-9436919-B2
Application number	US-201414473109-A
Country	US
Kind code	B2
Filing date	Aug 29, 2014
Priority date	Mar 28, 2013
Publication date	Sep 6, 2016
Grant date	Sep 6, 2016

Abstract

Systems and methods are disclosed herein for tuning an item classification. In one aspect, a method may include receiving, by a computing device, a classification request. The computing device may determine an item type of the item using a plurality of classifiers, and generate information of item type determination corresponding to the item. In response to a determination that a confidence score associated with the determined item type is less than a predetermined threshold, the computing device may provide the information of item type determination for analysis.

First claim

The invention claimed is:

1. A computer-implemented method for product classification, the method comprising:
receiving, by one or more processors of a server, a classification request comprising a description of a product;
determining, by the one or more processors, a type of the product using a plurality of classifiers;
determining, by the one or more processors, a confidence score associated with the type of the product;
determining, by the one or more processors, whether the confidence score associated with the type of the product is less than a predetermined threshold;
in response to a determination that the confidence score associated with the type of the product is less than the predetermined threshold, providing, by the one or more processors, the type of the product for analysis;
in response to the confidence score associated with the type of the product and the predetermined threshold, submitting the description of the product and the type of the product to a crowdsourcing forum for a validation decision; and
returning, from the crowdsourcing forum, the validation decision, wherein:
when the crowdsourcing forum determines that the type of the product is correct, the validation decision indicates that the type of the product is correct;
when the crowdsourcing forum determines that the type of the product is not correct, the validation decision indicates that the type of the product is not correct; and
when the crowdsourcing forum determines the description of the product is insufficient to accurately judge whether the type of the product is correct, the validation decision indicates that the type of the product is unclear.

2. The method of claim 1 , further comprising: in response to a determination that the confidence score associated with the type of the product is not less than the predetermined threshold, providing, by the one or more processors, the description of the product and the type of the product to the crowdsourcing forum for the validation decision.

3. The method of claim 2 , further comprising:
storing the description of the product and the type of the product; and
generating a report comprising the analysis or the validation decision.

4. The method of claim 2 , further comprising:
determining, based on the analysis or the validation decision, that the type of the product is not acceptable for the description of the product; and
causing generation, through an analyst computer, of a new rule and an updated type of the product.

5. The method of claim 4 , further comprising:
updating a rule-based classifier using the new rule; and
training a machine learning-based classifier using the updated type of the product, wherein the plurality of classifiers comprises the rule-based classifier and the machine learning-based classifier.

6. The method of claim 5 , further comprising:
generating a report of the type of the product for multiple different products based on the validation decision and the updated type of the product; and
providing the report to a user.

7. The method of claim 1 , wherein the description of the product comprises a suggested product type.

8. The method of claim 7 , further comprising:
determining that the type of the product comprises the suggested product type; and
providing the description of the product and the type of the product to the crowdsourcing forum for the validation decision.

9. The method of claim 1 , wherein the plurality of classifiers comprises a plurality of rule-based classifiers that implement a white-list rule and a blacklist rule for product classification, wherein the white-list rule defines that the product belongs to a first product type, and wherein the blacklist rule defines that the product does not belong to a second product type different from the first product type.

10. The method of claim 1 , wherein the plurality of classifiers comprises a naive Bayes classifier, a k-Nearest Neighbors (KNN) classifier, and a Perceptron classifier.

11. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed on one or more processors, cause the one or more processors to perform acts comprising:
receiving a classification request comprising a description of an item;
determining an item type of the item using a plurality of classifiers wherein the plurality of classifiers comprises a rule-based classifier that implements a white-list rule and a blacklist rule for item classification, wherein the white-list rule defines that the item belongs to a first item type, and wherein the blacklist rule defines that the item does not belong to a second item type different from the first item type;
generating information of item type determination corresponding to the item, the information of item type determination comprising the description of the item and the item type of the item;
determining whether a confidence score associated with the item type of the item is less than a predetermined threshold;
in response to the confidence score associated with the item type of the item and the predetermined threshold, submitting the information of item type determination to a crowdsourcing forum for a validation decision; and
returning, from the crowdsourcing forum, the validation decision, wherein:
when the crowdsourcing forum determines that the item type of the item is correct, the validation decision indicates that the item type of the item is correct;
when the crowdsourcing forum determines that the item type of the item is not correct, the validation decision indicates that the item type of the item is not correct; and
when the crowdsourcing forum determines that the information of item type determination is insufficient to accurately judge whether the item type of the item is correct, the validation decision indicates that the item type of the item is unclear.

12. The one or more non-transitory computer-readable media of claim 11 , wherein the acts further comprise:
providing the information of item type determination to an analyst computer for analysis by an analyst when the confidence score associated with the item type of the item is less than the predetermined threshold.

13. The one or more non-transitory computer-readable media of claim 12 , wherein, when the crowdsourcing forum determines that the information of item type determination is insufficient to accurately judge whether the item type of the item is correct, the acts further comprise:
causing generation of a new rule and a updated item type determination;
updating the rule-based classifier using the new rule; and
training a machine learning-based classifier using the updated item type determination, wherein the plurality of classifiers comprises the machine learning-based classifier.

14. The one or more non-transitory computer-readable media of claim 11 , wherein the plurality of classifiers comprises a plurality of rule-based classifiers that implement white-list rules and blacklist rules for item classification.

15. The one or more non-transitory computer-readable media of claim 11 , wherein the description of the item comprises a suggested item type, and wherein the acts further comprise:
determining that the item type of the item comprises the suggested item type; and
providing the information of item type determination to a crowdsourcing computer network for evaluation.

16. An apparatus comprising: a memory configured to store data and one or more sets of instructions; and one or more processors coupled to the memory, the one or more processors configured to execute the one or more sets

Assignees

• Wal Mart Stores Inc

Inventors

• Sun Chong
• Yang Fan
• Rampalli Narasimhan

Classifications

• G06Q10/40

  Business processes related to social networking or social networking services · CPC title

• G06Q10/101

  Collaborative creation, e.g. joint development of products or services · CPC title

• G06N99/005 Primary

  Physics · mapped topic

• G06Q50/01

  Physics · mapped topic

• G06F17/30707

  Physics · mapped topic