Item recommendations using image feature data

Patent metadata

Field	Value
Publication number	US-10489692-B2
Application number	US-201715661722-A
Country	US
Kind code	B2
Filing date	Jul 27, 2017
Priority date	Sep 30, 2011
Publication date	Nov 26, 2019
Grant date	Nov 26, 2019

Abstract

An apparatus and method to facilitate providing recommendations are disclosed herein. A networked system receives a query image, and performs image processing to extract a color histogram of an item depicted in the query image. The networked system identifies a dominant color of the item, whereby the dominant color is a color that is present on a most spatial area of the item relative to any other color of the item. The color histogram of the item is compared with a color histogram of an image of each of a plurality of available items, or the dominate color of the item is compared with a dominate color of the image of each of a plurality of available items. Based on the comparing, the networked system identifies item recommendations of the available items that closely match the item depicted in the query image, and the item recommendations are presented.

First claim

What is claimed is:

1. A method comprising:
receiving, at a networked system from an application installed on a device of a user, a query image captured by the device via the application;
performing, by the networked system, image processing to extract a color histogram of an item depicted in the query image;
identifying, by a hardware processor of the networked system, a dominant color of the item depicted in the query image, the dominant color being a color that is present on a most spatial area of the item relative to any other color of the item;
comparing, by the networked system, the color histogram of the item depicted in the query image with a color histogram of an image of each of a plurality of available items or the dominate color of the item depicted in the query image with a dominate color of the image of each of a plurality of available items;
based on the comparing, identifying, by the networked system, item recommendations of the available items that closely match the item depicted in the query image; and
causing presentation of the item recommendations on the device of the user.

2. The method of claim 1 , further comprising determining a dominate color confidence score using the color histogram of the query image.

3. The method of claim 2 , wherein the comparing comprises:
comparing, by the networked system, the color histogram of the item depicted in the query image with the color histogram of the image of each of the plurality of available items based on the dominate color confidence score being low, or
comparing, by the networked system, the dominate color of the item depicted in the query image with a dominate color of the image of each of a plurality of available items based on the dominate color confidence score being high.

4. The method of claim 1 , wherein the identifying the dominant color comprises re-using hue values from the color histogram and identifying a bin within a bin group corresponding to the hue values that has a highest peak.

5. The method of claim 1 , further comprising providing the application for installation on the device of the user, the application including camera capabilities that configure the device to capture the query image, the application transmitting the query image to the networked system.

6. The method of claim 1 , further comprising:
receiving, by the networked system from the remote device, a selection of a recommendation from the item recommendations; and
in response to receiving the selection of the recommendation, causing display of item details and an image of the selected recommendation on the user interface of the remote device.

7. The method of claim 6 , further comprising, in response to receiving the selection of the recommendation,
determining a set of refined recommendations; and
causing display of the set of refined recommendations on the user interface displaying the item details and the image of the selected recommendation.

8. A system comprising:
one or more hardware processors; and
a memory storing instructions that, when executed by one or more hardware processors, causes the one or more hardware processors to perform operations comprising:
receiving, from an application installed on a device of a user, a query image captured by the device via the application;
performing image processing to extract a color histogram of an item depicted in the query image;
identifying a dominant color of the item depicted in the query image, the dominant color being a color that is present on a most spatial area of the item relative to any other color of the item;
comparing the color histogram of the item depicted in the query image with a color histogram of an image of each of a plurality of available items or the dominate color of the item depicted in the query image with a dominate color of the image of each of a plurality of available items;
based on the comparing, identifying item recommendations of the available items that closely match the item depicted in the query image; and
causing presentation of the item recommendations on the device of the user.

9. The system of claim 8 , wherein the operations further comprise determining a dominate color confidence score using the color histogram of the query image.

10. The system of claim 9 , wherein the comparing comprises:
comparing, by the networked system, the color histogram of the item depicted in the query image with the color histogram of the image of each of the plurality of available items based on the dominate color confidence score being low, or
comparing, by the networked system, the dominate color of the item depicted in the query image with a dominate color of the image of each of a plurality of available items based on the dominate color confidence score being high.

11. The system of claim 8 , wherein the identifying the dominant color comprises re-using hue values from the color histogram and identifying a bin within a bin group corresponding to the hue values that has a highest peak.

12. The system of claim 8 , wherein the operations further comprise providing the application for installation on the device of the user, the application including camera capabilities that configure the device to capture the query image, the application transmitting the query image to the networked system.

13. The system of claim 8 , wherein the operations further comprise:
receiving, by the networked system from the remote device, a selection of a recommendation from the item recommendations; and
in response to receiving the selection of the recommendation, causing display of item details and an image of the selected recommendation on the user interface of the remote device.

14. The system of claim 13 , wherein the operations further comprise, in response to receiving the selection of the recommendation,
determining a set of refined recommendations; and
causing display of the set of refined recommendations on the user interface displaying the item details and the image of the selected recommendation.

15. A non-transitory machine-readable medium storing instructions that, when executed by one or more hardware processors of a machine, causes the machine to perform operations comprising:
receiving, from an application installed on a device of a user, a query image captured by the device via the application;
performing image processing to extract a color histogram of an item depicted in the query image;
identifying a dominant color of the item depicted in the query image, the dominant color being a color that is present on a most spatial area of the item relative to any other color of the item;
comparing the color histogram of the item depicted in the query image with a color histogram of an image of each of a plurality of available items or the dominate color of the item depicted in the query image with a dominate color of the image of each of a plurality of available items;
based on the comparing, identifying item recommendations of the available items that closely match the item depicted in the query image; and
causing presentation of the item recommendations on the device of the user.

16. The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise determining a dominate color confidence score using the color histogram of the query image.

17. The non-transitory machine-readable medium of claim 16 , wherein the comparing comprises:
comparing, by the networked system, the color histogram of the item depicted in the query image with the color histogram of the image of each of the plurality of avai

Assignees

• Ebay Inc

Inventors

• Bhardwaj Anurag
• Di Wei
• Jagadeesh Vignesh
• Piramuthu Robinson
• Sundaresan Neelakantan

Classifications

• G06T7/90 Primary

  Determination of colour characteristics · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F18/21

  Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation · CPC title

• G06F16/532

  Query formulation, e.g. graphical querying · CPC title

• G06Q30/0631

  Recommending goods or services · CPC title