Image analysis and prediction based visual search

Patent metadata

Field	Value
Publication number	US-2018107902-A1
Application number	US-201615294773-A
Country	US
Kind code	A1
Filing date	Oct 16, 2016
Priority date	Oct 16, 2016
Publication date	Apr 19, 2018
Grant date	—

Abstract

Methods, systems, and computer programs are presented for adding new features to a network service. A method includes receiving an image depicting an object of interest. A category set is determined for the object of interest and an image signature is generated for the image. Using the category set and the image signature, the method identifies a set of publications within a publication database and assigns a rank to each publication. The method causes presentation of the ranked list of publications at a computing device from which the image was received.

First claim

What is claimed is:

1 . A method, comprising:
receiving, by at least one processor of a server, at least one image depicting at least a portion of an object of interest; determining, by the at least one processor, a category set for the object of interest; generating, by the at least one processor, an image signature for the at least one image, the image signature comprising a vector representation of the at least one image; using the category set and the image signature for the at least one image, identifying, by the at least one processor, a set of publications within a publication database; assigning a rank to each publication of the set of publications based on the image signature to generate a ranked list of publications; and causing presentation of the ranked list of publications at a computing device from which the at least one image was received.

2 . The method of claim 1 , further comprising:
identifying, by the at least one processor, a set of aspects representing one or more attributes of the object of interest within the at least one image; and for each aspect of the set of aspects, determining a probability the object of interest within the at least one image includes a specified aspect to generate a confidence score for each aspect.

3 . The method of claim 2 , wherein the one or more attributes are elements of an appearance of the object of interest and each aspect, of the set of aspects, is a descriptive word associated with a specified attribute.

4 . The method of claim 2 , wherein the ranked list is organized according to a first rank order, the method further comprising:
for each publication of the set of publications, identifying a set of metadata descriptors; generating an aspect ranking score for each publication of the set of publications, the aspect ranking score generated by performing a weighted comparison of the set of aspects of the object of interest and the set of metadata descriptors; and generating a modified ranked list of publications organized according to a second rank order reflecting a combination of the aspect ranking scores and the ranks based on the image signature.

5 . The method of claim 1 , wherein determining the category set for the object of interest further comprises:
identifying, by the at least one processor, a set of aspects representing one or more attributes of the object of interest within the at least one image; and determining one or more categories associated with at least one aspect of the set of aspects for inclusion in the category set.

6 . The method of claim 1 , wherein using the category set and the image signature to identify the set of publications further comprises:
selecting query publications associated with one or more category of the category set; comparing the image signature for the at least one image with a set of image signatures associated with the query publications to determine one or more similar image signatures; and identifying the set of publications as a subset of the query publications associated with the one or more similar image signatures.

7 . The method of claim 1 , wherein the at least one image is a set of frames comprising a video, the method further comprising:
determining, by the at least one processor, a first category set for the object of interest in a first image and a second category set for the object of interest in a second image, the first image and the second image being selected from the set of frames of the video; generating, by the at least one processor, a first image signature comprising a first vector representation of the first image and a second image signature comprising a second vector representation of the second image; using the first category set, the second category set, the first image signature, and the second image signature, identifying, by the at least one processor, the set of publications within the publication database; and assigning a rank to each publication of the set of publications based on one or more of the first image signature and the second image signature to generate a ranked list of publications.

8 . A system, comprising:
one or more hardware processors; and a non-transitory machine-readable storage medium including instructions that, when executed by the one or more hardware processors, cause the one or more hardware processors to perform operations comprising:
receiving, by the one or more processor of a server, at least one image depicting at least a portion of an object of interest;
determining, by the one or more processor, a category set for the object of interest;
generating, by the one or more processor, an image signature for the at least one image, the image signature comprising a vector representation of the at least one image;
based on the category set and the image signature for the at least one image, identifying, by the one or more processor, a set of publications within a publication database;
assigning a rank to each publication of the set of publications based on the image signature to generate a ranked list of publications; and
causing presentation of the ranked list of publications at a computing device from which the at least one image was received.

9 . The system of claim 8 , wherein the operations further comprise:
identifying, by the one or more processor, a set of aspects representing one or more attributes of the object of interest within the at least one image; and for each aspect of the set of aspects, determining a probability the object of interest within the at least one image includes a specified aspect to generate a confidence score for each aspect.

10 . The system of claim 9 , wherein the one or more attributes are elements of an appearance of the object of interest and each aspect, of the set of aspects, is a descriptive word associated with a specified attribute.

11 . The system of claim 9 , wherein the ranked list is organized according to a first rank order, the operations further comprising:
for each publication of the set of publications, identifying a set of metadata descriptors; generating an aspect ranking score for each publication of the set of publications, the aspect ranking score generated by performing a weighted comparison of the set of aspects of the object of interest and the set of metadata descriptors; and generating a modified ranked list of publications organized according to a second rank order reflecting a combination of the aspect ranking scores and the ranks based on the image signature.

12 . The system of claim 8 , wherein determining the category set for the object of interest further comprises:
identifying, by the one or more processor, a set of aspects representing one or more attributes of the object of interest within the at least one image; and determining one or more categories associated with at least one aspect of the set of aspects for inclusion in the category set.

13 . The system of claim 8 , wherein using the category set and the image signature to identify the set of publications further comprises:
selecting query publications associated with one or more category of the category set; comparing the image signature for the at least one image with a set of image signatures associated with the query publications to determine one or more similar image signatures; and identifying the set of publications as a subset of the query publications associated with the one or more similar image signatures.

14 . The system of claim 8 , wherein the at least one image is a set of frames comprising a video, the operations further comprising:
determining, by the one o

Assignees

• Ebay Inc

Inventors

• Yang Fan
• Kale Ajinkya Gorakhnath
• Wang Qiaosong
• Kiapour Mohammadhadi
• Piramuthu Robinson

Classifications

• G06V10/771

  Feature selection, e.g. selecting representative features from a multi-dimensional feature space · CPC title

• G06F18/2415

  based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate · CPC title

• G06F16/532 Primary

  Query formulation, e.g. graphical querying · CPC title

• G06F18/2113

  by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title