Parallel prediction of multiple image aspects

Patent metadata

Field	Value
Publication number	US-12405997-B2
Application number	US-202318200460-A
Country	US
Kind code	B2
Filing date	May 22, 2023
Priority date	Oct 16, 2016
Publication date	Sep 2, 2025
Grant date	Sep 2, 2025

Abstract

Example embodiments that analyze images to characterize aspects of the images rely on a same neural network to characterize multiple aspects in parallel. Because additional neural networks are not required for additional aspects, such an approach scales with increased aspects.

First claim

What is claimed is:

1. A method comprising:
receiving, by a network-based marketplace from a user device, a search query for an item listing of the network-based marketplace, the search query including an input image depicting an item;
identifying, by a machine learning system, matches between an input semantic vector representing the input image and one or more image vectors of one or more images from a publication corpus;
identifying a probability that the input image is associated with an aspect value that characterizes the item depicted in the input image based at least in part on the one or more image vectors;
adding metadata comprising the aspect value to the input image responsive to the probability that the input image is associated with the aspect value exceeding a threshold; and
transmitting, to the user device, a set of one or more search results responsive to the search query, wherein the set of one or more search results are identified based at least in part on the metadata and the input image.

2. The method of claim 1 , further comprising:
receiving one or more aspect probabilities from the machine learning system, the one or more aspect probabilities identifying a respective probability that the input image has a respective aspect value of one or more aspect values characterizing the input image, the one or more aspect values comprising the aspect value.

3. The method of claim 2 , wherein receiving, from the machine learning system, the one or more aspect probabilities comprises:
receiving the one or more aspect probabilities as parallel outputs from the machine learning system, wherein the parallel outputs include output location data with a location within a respective image.

4. The method of claim 1 , further comprising:
determining that one or more aspect values characterizing the input image are missing, the one or more aspect values comprising a category of the item depicted in the input image, a color of the item depicted in the input image, or both, wherein identifying matches between the input semantic vector representing the input image and the one or more image vectors is based at least in part on the determining.

5. The method of claim 4 , further comprising:
generating the input semantic vector based at least in part on the determining.

6. The method of claim 1 , wherein the matches are identified based on a measure of similarity between the input semantic vector and the one or more image vectors.

7. The method of claim 1 , further comprising:
ranking search results of the set of one or more search results based at least in part on the aspect value.

8. The method of claim 1 , further comprising:
determining a product category for the item based on the aspect value, wherein the set of one or more search results are based on the product category.

9. A system comprising:
one or more hardware processors; and
a memory storing instructions that, when executed by the one or more hardware processors, cause the system to perform operations comprising:
receiving, from a user device, a search query for an item listing of a network-based marketplace, the search query including an input image depicting an item;

identifying, by a machine learning system, matches between an input semantic vector representing the input image and one or more image vectors of one or more images from a publication corpus;
identifying a probability that the input image is associated with an aspect value that characterizes the item depicted in the input image based at least in part on the one or more image vectors;
adding metadata comprising the aspect value to the input image responsive to the probability that the input image is associated with the aspect value exceeding a threshold; and
transmitting, to the user device, a set of one or more search results responsive to the search query, wherein the set of one or more search results are identified based at least in part on the metadata and the input image.

10. The system of claim 9 , the operations further comprising:
receiving one or more aspect probabilities from the machine learning system, the one or more aspect probabilities identifying a respective probability that the input image has a respective aspect value of one or more aspect values characterizing the input image, the one or more aspect values comprising the aspect value.

11. The system of claim 10 , wherein the instructions for receiving the one or more aspect probabilities, when executed by the one or more hardware processors, further cause the system to perform operations comprising receiving the one or more aspect probabilities as parallel outputs from the machine learning system, wherein the parallel outputs include output location data with a location within a respective image.

12. The system of claim 9 , the operations further comprising:
determining that one or more aspect values characterizing the input image are missing, the one or more aspect values comprising a category of the item depicted in the input image, a color of the item depicted in the input image, or both, wherein identifying matches between the input semantic vector representing the input image and the one or more image vectors is based at least in part on the determining.

13. The system of claim 12 , the operations further comprising:
generating the input semantic vector based at least in part on the determining.

14. The system of claim 9 , wherein the matches are identified based on a measure of similarity between the input semantic vector and the one or more image vectors.

15. The system of claim 9 , the operations further comprising:
ranking search results of the set of one or more search results based at least in part on the aspect value.

16. The system of claim 9 , the operations further comprising:
determining a product category for the item based on the aspect value, wherein the set of one or more search results are based on the product category.

17. A non-transitory computer readable storage medium comprising instructions that when executed by one or more processors cause a system to perform operations comprising:
receiving, from a user device, a search query for an item listing of a network- based marketplace, the search query including an input image depicting an item;
identifying, by a machine learning system, matches between an input semantic vector representing the input image and one or more image vectors of one or more images from a publication corpus;
identifying a probability that the input image is associated with an aspect value that characterizes the item depicted in the input image based at least in part on the one or more image vectors;
adding metadata comprising the aspect value to the input image responsive to the probability that the input image is associated with the aspect value exceeding a threshold; and
transmitting, to the user device, a set of one or more search results responsive to the search query, wherein the set of one or more search results are identified based at least in part on the aspect value metadata and the input image.

18. The non-transitory computer readable storage medium of claim 17 , the operations further comprising:
receiving one or more aspect probabilities from the machine learning system, the one or more aspect probabilities identifying a respective probability that the input image has a respective aspect value of one or more aspect values characterizing the input image, the one or more aspect values comprising the aspect value.

19. The method of claim 1 , wherei

Assignees

• Ebay Inc

Inventors

• Kiapour Mohammadhadi
• Kale Ajinkya Gorakhnath
• Piramuthu Robinson
• Yu Licheng

Classifications

• G06F16/583 Primary

  using metadata automatically derived from the content · CPC title

• G06F16/58 Primary

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• H04L67/10

  in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) · CPC title