System and method for determining substitutes for a requested product and the order to provide the substitutes

Patent metadata

Field	Value
Publication number	US-11373231-B2
Application number	US-201916264323-A
Country	US
Kind code	B2
Filing date	Jan 31, 2019
Priority date	Jan 31, 2019
Publication date	Jun 28, 2022
Grant date	Jun 28, 2022

Abstract

A system can include one or more processors and non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform a method for determining one or more substitutes for a product requested in an inquiry from a user and the order of the one or more substitutes to be transmitted to the user. The method can comprise: receiving an inquiry from a user, wherein the inquiry comprises an image and a text describing a requested product; extracting one or more visual attributes of the requested product based on the image; extracting one or more textual attributes of the requested product based on the text; determining a first projected requested product, in a product graph relating to a domain of the requested product, based on the one or more visual attributes of the requested product; determining a second projected requested product in the product graph based on the one or more textual attributes of the requested product; and determining one or more substitutes for the requested product. In this embodiment, the one or more substitutes for the requested product can be determined by: determining one or more first candidates, each first candidate of the one or more first candidates existing in both the product graph and an inventory database and comprising a visual distance between the first projected requested product and the each first candidate in the product graph; determining one or more second candidates, each second candidate of the one or more second candidates existing in both the product graph and the inventory database and comprising a textual distance between the second projected requested product and the each second candidate in the product graph, the one or more substitutes comprising the one or more first candidates and the one or more second candidates; determining a score of a candidate substitute of the one or more substitutes based on a visual distance and a textual distance of the candidate substitute of the one or more substitutes; and sorting the one or more substitutes based on the score of the candidate substitute of the one or more substitutes. In this embodiment, after the one or more substitutes are determined and sorted, this method can further comprise: in response to receiving the inquiry from the user, automatically transmitting to the user higher ranked ones of the one or more substitutes. Additionally, in this embodiment, the product graph can be dynamically generated by a machine learning module based on information about the domain of the requested product; and the product graph can comprise: (a) one or more product nodes, the one or more product nodes comprising the one or more substitutes; (b) one or more entity nodes, the one or more entity nodes comprising the one or more visual attributes and the one or more textual attributes of the requested product; (c) one or more entity-entity connections between each pair of related entity nodes of the one or more entity nodes; (d) one or more product-entity connections between a product node of the one or more product nodes and an entity node of the one or more entity nodes, wherein the entity node is related to the product node; and (e) one or more product-product connection between a first product node of the one or more product nodes and a second product node of the one or more product nodes, wherein the first product node and the second product node are determined to be substitutable by or complimentary of each other. Other embodiments are disclosed.

First claim

What is claimed is:

1. A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform:
dynamically training a machine learning module to build a product graph relating to a domain;
receiving, via computer network, an inquiry from a user device for a user, wherein the inquiry comprises an image and a text describing a requested product in the domain;
extracting one or more visual attributes of the requested product based on the extracting one or more textual attributes of the requested product based on the text;
determining a first projected requested product, in the product graph, based on the one or more visual attributes of the requested product;
determining a second projected requested product in the product graph based on the one or more textual attributes of the requested product;
determining one or more substitutes for the requested product by:
determining one or more first candidates, each first candidate of the one or more first candidates existing in both the product graph and an inventory database and comprising a visual distance between the first projected requested product and the each first candidate in the product graph;
determining one or more second candidates, each second candidate of the one or more second candidates existing in both the product graph and the inventory database and comprising a textual distance between the second projected requested product and the each second candidate the product graph, the one or more substitutes comprising the one or more first candidates and the one or more second candidates;
determining a score of a candidate substitute of the one or more substitutes based on the visual distance and the textual distance of the candidate substitute of the one or more substitutes, Wherein a greater weight is given to the visual distance than the textual distance in determining the score; and
sorting the one or more substitutes based on the score of the candidate substitute of the one or more substitutes; and

in response to receiving the inquiry from the user, automatically transmitting, via the computer network and to the user device for the user, information about higher ranked ones of the one or more substitutes to facilitate a display of the information on a graphical user interface on the user device,
wherein:
the product graph comprises:
a multi-dimensional space;
one or more product nodes, wherein:
the one or more product nodes comprise the one or more substitutes; and
each of the one or more product nodes comprises a respective product coordinate in the multi-dimensional space;

one or more entity nodes, wherein:
the one or more entity nodes comprise the one or more visual attributes and the one or more textual attributes of the requested product; and
each of the one or more entity nodes comprises a respective entity coordinate in the multi-dimensional space;

one or more entity-entity connections between each pair of related entity nodes of the one or more entity nodes;
one or more product-entity connections, wherein:
each of the one or more product-entity connections is between a product node of the one or more product nodes and an entity node of the one or more entity nodes;
the entity node is related to the product node; and
the each of the one or more product-entity connections comprises a respective product-entity connection distance determined based on a respective relatedness degree between the product node and the entity node; and

one or more product-product connections between a first product node of the one or more product nodes and a second product node of the one or more product nodes, wherein the first product node and the second product node are determined to be substitutable by or complimentary of each other;

determining the one or more first candidates further comprises determining the visual distance between the first projected requested product and each first candidate of the one or more first candidates in the product graph based on a respective first vector between the first projected requested product and the each first candidate as projected in each dimension of multiple dimensions of the multi-dimensional space; and
determining the one or more second candidates further comprises determining the textual distance between the second projected requested product and each second candidate of the one or more second candidates in the product graph based on a respective second vector between the second projected requested product and the each second candidate as projected in each dimension of the multiple dimensions of the multi-dimensional space.

2. The system in claim 1 , wherein the computing instructions are further configured to run on the one or more processors and perform:
extracting one or more color attributes of the requested product based on the image and the text; and
determining a color distance of the candidate substitute of the one or more substitutes based on the one or more color attributes of the requested product and one or more color entities of the candidate substitute of the one or more substitutes;
wherein:
the one or more entity nodes of the product graph further comprise the one or more color attributes of the requested product and the one or more color entities of the candidate substitute of the one or more substitutes; and
the determining the score of the candidate substitute of the one or more substitutes further comprises determining the score of the candidate substitute of the one or more substitutes based on the color distance of the candidate substitute of the one or more substitutes.

3. The system in claim 2 , wherein:
the one or more color attributes of the requested product comprises a product major color and a product minor color; and
the one or more color entities of the candidate substitute of the one or more substitutes comprise a candidate major color and a candidate minor color.

4. The system in claim 3 , wherein:
the score of the candidate substitute of the one or more substitutes is determined by:

α
I

⁢

1
d

+

β
C

⁡

(

1

d

m
⁢
a
⁢
j

-
c

+

1

d

m
⁢
t
⁢
n

-
c

)

+

Assignees

• Walmart Apollo Llc

Inventors

• Soohoo Anthony
• Raghavendra Vijay
• Myslinsky Charles
• Kochar Harman
• Amarthaluri Abilash
• Gupta Ashish
• Deivasigamani Karthik
• Ahmadi Behzad
• Kenkre Sreyash D
• Deep Rohit
• Nafar Zuzar Fakhruddin
• Ovenc Omer
• Kandaswamy Venkatesh
• Kumaran Arvind Senthil
• Ashfaq Suleman

Classifications

• G06V20/70

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title

• G06V20/00

  Scenes; Scene-specific elements (control of digital cameras H04N23/60) · CPC title

• G06V10/84

  using probabilistic graphical models from image or video features, e.g. Markov models or Bayesian networks · CPC title

• G06Q30/0631 Primary

  Recommending goods or services · CPC title

• G06Q10/087 Primary

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title