Image analysis and prediction based visual search

Patent metadata

Field	Value
Publication number	US-11914636-B2
Application number	US-202217949953-A
Country	US
Kind code	B2
Filing date	Sep 21, 2022
Priority date	Oct 16, 2016
Publication date	Feb 27, 2024
Grant date	Feb 27, 2024

Abstract

Methods, systems, and computer programs are presented for adding new features to a network service. A method includes receiving an image depicting an object of interest. A category set is determined for the object of interest and an image signature is generated for the image. Using the category set and the image signature, the method identifies a set of publications within a publication database and assigns a rank to each publication. The method causes presentation of the ranked list of publications at a computing device from which the image was received.

First claim

What is claimed is:

1. A system comprising:
at least one processor; and
at least one memory including instructions which when executed by the at least one processor causes the system to perform operations comprising:
receiving a video comprising a set of frames from a user device;
generating a composite image of two or more frames of the video;
determine a category set of the composite image;
identifying a set of publications from a publications database;
generating an image signature for the composite image;
assigning a rank to each publication of the set of publications based at least in part on a comparison of the image signature of the composite image to one or more respective image signatures associated with a respective publication; and
causing, at the user device, presentation of a ranked list of the set of publications based at least in part on the rank of each publication of the set of publications.

2. The system of claim 1 , wherein identifying the set of publications comprises:
identifying one or more categories of the publications database that match the category set; and
comparing respective image signatures of publications within the one or more categories with the image signature for the composite image to identify the set of publications.

3. The system of claim 1 , wherein generating the image signature comprises: generating a binary vector representation of the composite image via a hashing layer of a neural network, wherein the image signature comprises the binary vector representation.

4. The system of claim 1 , wherein the instructions when executed by the at least one processor cause the system to perform further operations comprising:
determining a respective Hamming distance between the image signature and respective image signatures of each of the set of publications, wherein assigning the rank to each publication of the set of publications is based at least in part on the respective Hamming distances.

5. The system of claim 1 , wherein the instructions when executed by the at least one processor cause the system to perform further operations comprising:
causing, at the user device, presentation of a product listing for an electronic commerce system associated with a publication of the set of publications.

6. The system of claim 1 , wherein determining the category set comprises:
identifying a set of publication categories associated with the publications database;
generating a semantic vector for the composite image; and
comparing the semantic vector to respective semantic vectors associated with each publication category of the set of publication categories.

7. The system of claim 6 , wherein the semantic vector comprises a set of descriptive words associated with the composite image.

8. The system of claim 1 , wherein the instructions when executed by the at least one processor cause the system to perform further operations comprising:
causing, at the user device, presentation of one or more user interface elements enabling access of an image capture device of the user device and initiation of one or more processes to capture the video.

9. The system of claim 8 , wherein receiving the video comprises receiving the video in real time as the video is captured by the image capture device.

10. The system of claim 1 , wherein the instructions when executed by the at least one processor cause the system to perform further operations comprising:
causing, at the user device, presentation of one or more user interface elements enabling selection of the video from a storage location at the user device.

11. The system of claim 1 , wherein generating the composite image comprises:
incorporating a plurality of visual attributes, aspects, and characteristics of the two or more frames into the composite image.

12. A method comprising:
receiving a video comprising a set of frames from a user device;
generating, via hardware processing circuitry, a composite image of two or more frames of the video;
determine, via the hardware processing circuitry, a category set of the composite image;
identifying, based at least in part on the category set, a set of publications from a publications database;
generating, via the hardware processing circuitry, an image signature for the composite image;
assigning a rank to each publication of the set of publications based at least in part on a comparison of the image signature of the composite image to one or more respective image signatures associated with a respective publication; and
causing, at the user device, presentation of a ranked list of the set of publications based at least in part on the rank of each publication of the set of publications.

13. The method of claim 12 , wherein identifying the set of publications comprises:
identifying, by the hardware processing circuitry, one or more categories of the publications database that match the category set; and
comparing, by the hardware processing circuitry, respective image signatures of publications within the one or more categories with the image signature for the composite image to identify the set of publications.

14. The method of claim 12 , wherein generating the image signature comprises:
generating, by the hardware processing circuitry, a binary vector representation of the composite image via a hashing layer of a neural network, wherein the image signature comprises the binary vector representation.

15. The method of claim 12 , further comprising:
determining, by the hardware processing circuitry, a respective Hamming distance between the image signature and respective image signatures of each of the set of publications, wherein assigning the rank to each publication of the set of publications is based at least in part on the respective Hamming distances.

16. The method of claim 12 , further comprising:
causing, at the user device, presentation of a product listing for an electronic commerce system associated with a publication of the set of publications.

17. The method of claim 12 , wherein determining the category set comprises:
identifying, by the hardware processing circuitry, a set of publication categories associated with the publications database;
generating, by the hardware processing circuitry, a semantic vector for the composite image; and
comparing, by the hardware processing circuitry, the semantic vector to respective semantic vectors associated with each publication category of the set of publication categories.

18. The method of claim 17 , wherein the semantic vector comprises a set of descriptive words associated with the composite image.

19. The method of claim 12 , further comprising:
causing, at the user device, presentation of one or more user interface elements enabling access of an image capture device of the user device and initiation of one or more processes to capture the video.

20. A non-transitory computer-readable storage medium including program code which when executed by at least one processor causes a system to perform operations comprising:
receiving a video comprising a set of frames from a user device;
generating a composite image of two or more frames of the video;
determine a category set of the composite image;
identifying a set of publications from a publications database;
generating an image signature for the composite image;
assigning a rank to each publication of the set of publications based at least in part on a comparison of the image signature of the composite image to one or more respective ima

Assignees

• Ebay Inc

Inventors

• Yang Fan
• Kale Ajinkya Gorakhnath
• Wang Qiaosong
• Kiapour Mohammadhadi
• Piramuthu Robinson

Classifications

• G06F16/55

  Clustering; Classification · CPC title

• G06F16/53

  Querying · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title