Visual quality performance predictors

Patent metadata

Field	Value
Publication number	US-12002083-B2
Application number	US-202117448083-A
Country	US
Kind code	B2
Filing date	Sep 20, 2021
Priority date	Jul 6, 2021
Publication date	Jun 4, 2024
Grant date	Jun 4, 2024

Abstract

A visual search recommendation engine utilizes an image quality indication model for a visual search recommendation. Specifically, the visual search recommendation engine receives a search image as a search query at a search engine. The visual search recommendation engine provides the search image as an input into the image quality indication model, which is trained to output an image quality indication based on image aspects of the search image. A plurality of images are identified from an image corpus. The visual search recommendation engine determines an image similarity based on a comparison between the plurality of images from the image corpus and the search image. The image quality indication and the image similarity indicate a search query performance for the search image. A first image exceeding the search query performance is identified. The first image is provided for display at the search engine.

First claim

What is claimed is:

1. One or more non-transitory computer storage media storing computer-readable instructions that when executed by a processor, cause the processor to perform operations for a visual search recommendation, the operations comprising:
receiving a search image as a search query at a search engine;
providing the search image as an input into an image quality indication model, the image quality indication model trained to output an image quality indication based on image aspects of the search image;
identifying a plurality of images from an image corpus;
determining a search query performance for the search query based on the image quality indication and an image similarity between each of the plurality of images and the search image;
identifying a first image exceeding the search query performance indicated by the image quality indication and the image similarity;
providing for display at the search engine, the first image as a recommendation for the search query;
receiving a selection of the first image recommended;
providing for display at the search engine, a set of search result images corresponding to the selection of the first image, the set of search result images exceeding the search query performance of the search image; and
receiving a selection of one of the set of search result images.

2. The media of claim 1 , wherein identifying the plurality of images from the image corpus further comprises:
receiving a selection of a category at the search engine; and
identifying images within the image corpus that are associated with the selected category, wherein the image similarity is determined from the images identified as associated with the selected category.

3. The media of claim 1 , further comprising:
prior to determining the search query performance, retrieving a first set of search results for the search image from the image corpus;
prior to determining the search query performance, selecting a subset of images from the first set of search results based on a ranking of the first set of search results; and
determining a post-retrieval image similarity based on a comparison between the subset of images and the search image, wherein the search query performance of the search image is determined further based on the post-retrieval image similarity.

4. The media of claim 1 , further comprising:
identifying a second image that exceeds the search query performance of the search image; and
providing for display at the search engine, the second image as another recommendation for the search query, the first image and the second image being presented as selectable options.

5. The media of claim 4 , wherein the search image comprises a first aspect and a second aspect, and wherein the first image comprises the first aspect and the second image comprises the second aspect.

6. The media of claim 5 , wherein each image of the set of the search result images includes the first aspect associated with the first image.

7. The media of claim 1 , further comprising training the image quality indication model to output the image quality indication based on a training dataset comprising labeled images, the labeled images being labeled based on one or more of angle of view, blurriness, and background quality.

8. The media of claim 1 , wherein the trained image quality indication model is a convolutional neural network.

9. A computer implemented method for a visual search recommendation, the method comprising:
training an image quality indication model to output image quality indications of input images based on one or more of an angle of view, blurriness, and background quality;
generating an image similarity determination model, wherein the image similarity determination model determines an image similarity between a search image and images of an image corpus based on image latent vectors;
generating an output engine configured to output a recommended image for recommendation as a search query, the recommended image based on identifying an image exceeding a search query performance for a search image determined based on an image quality indication of the recommended image determined using the trained image quality indication model and the image similarity determined using the image similarity determination model;
in response to receiving a first search image as a first search query, providing a first recommended image, via the output engine, for the recommendation as the first search query based on the first recommended image exceeding the search query performance determined based on providing the first search image to the trained image quality indication model and the image similarity determination model;
receiving an indication to use the first recommended image;
providing a set of the search result images in response to receiving the indication to use the first recommended image, each of the set of the search result images exceeding the search query performance for the first search query; and
receiving a selection of one of the set of search result images.

10. The method of claim 9 , the output engine further configured to output the recommended image based on the image similarity of the recommended image having a maximum image similarity compared to image similarities.

11. The method of claim 10 , wherein the image similarity determination model comprises a convolutional neural network, the convolutional neural network trained to determine the image similarity.

12. The method of claim 9 , wherein the image quality indication model is trained to output the image quality indications of the input images based on a resolution.

13. The method of claim 12 , wherein the image quality indication model is trained additionally based on an item size, or an image size.

14. The method of claim 9 , wherein the image similarity determination model determines the image similarity based on intensity patterns within the images, and wherein the images are selected for determining the image similarity based on each of the images being with a same category.

15. A system for a visual search recommendation, the system comprising:
at least one processor; and
one or more computer storage media storing computer-readable instructions that when executed by the at least one processor, cause the at least one processor to perform operations comprising:
providing a search image at a search engine;
receiving a recommended image as a selectable option from the search engine based on the search image, wherein the recommended image is determined based on the recommended image having a search query performance exceeding a search query performance of the search image, the search query performance of the recommended image and the search query performance of the search image determined from an image quality indication determined by a trained image quality indication model and an image similarity determined by a trained image similarity determination model;
selecting the recommended image for use as a search query at the search engine;
receiving from the search engine a search results page comprising search result images corresponding to the recommended image selected the search result images exceeding the search query performance of the search image, the search result images being presented as selectable options; and
selecting one of the selectable options.

16. The system of claim 15 , wherein the trained image quality indication model is a convolutional neural network.

17. The system of claim 15 , further comprising:
receiving a set of search results identified for the

Assignees

• Ebay Inc

Inventors

• Guy Ido
• Novgorodov Slava
• Dagan Arnon

Classifications

• G06Q30/0633 Primary

  Managing shopping lists, e.g. compiling or processing purchase lists (shipping orders G06Q10/083; order filling G06Q10/087) · CPC title

• G06F11/3409

  for performance assessment · CPC title

• G06F16/24578

  using ranking · CPC title

• G06F16/532

  Query formulation, e.g. graphical querying · CPC title

• G06F16/9532

  Query formulation · CPC title