Image-based faceted system and method

Patent metadata

Field	Value
Publication number	US-10102227-B2
Application number	US-201815924923-A
Country	US
Kind code	B2
Filing date	Mar 19, 2018
Priority date	Jun 10, 2013
Publication date	Oct 16, 2018
Grant date	Oct 16, 2018

Abstract

Disclosed herein is a system and method that facilitate searching and/or browsing of images by clustering, or grouping, the images into a set of image clusters using facets, such as without limitation visual properties or visual characteristics, of the images, and representing each image cluster by a representative image selected for the image cluster. A map-reduce based probabilistic topic model may be used to identify one or more images belonging to each image cluster and update model parameters.

First claim

The invention claimed is:

1. A method comprising:
determining, by a computing device, an image gradient representation of each digital image of a plurality of digital images, the image gradient representation identifying a plurality of image gradients determined for the digital image;
building, by the computing device, a vocabulary of words using the image gradient representation of each digital image of the plurality of digital images;
determining, by the computing device and for each digital image of the plurality, a frequency of occurrence of each word, of the vocabulary of words, using the image gradient representation of the digital image;
generating, by the computing device and using the vocabulary, a probabilistic topic model comprising image-specific parameters for each digital image in the plurality of digital images and cluster-specific parameters for each cluster of a plurality of clusters;
assigning, by the computing device, each digital image from the plurality of digital images to a cluster of the plurality of clusters using the digital image's image-specific parameters, the frequency of occurrence of each word associated with the digital image and the probabilistic topic model, each cluster corresponding to a word in the vocabulary;
identifying, by the computing device and for each cluster of the plurality of clusters, a representative digital image for the cluster;
receiving, by the computing device, a digital image search request of a user;
generating, by the computing device, a set of words, from the vocabulary of words, for the received digital image;
identifying, via the computing device and using the set of words generated for the received image as a query, a number of representative digital images, each representative digital image of the number being representative of a cluster, of the plurality of clusters, associated with at least one word of the set of words generated for the received image; and
communicating, via the computing device, the number of representative digital images to the user for display on a device of the user.

2. The method of claim 1 , determination of the frequency of occurrence further comprising:
determining, by the computing device and for each digital image of the plurality, a frequency of occurrence for each word of the vocabulary of words using the image gradient representation of the digital image.

3. The method of claim 1 , image gradient representation determination further comprising:
partitioning a digital image, of the plurality, into a plurality of partitions;
identifying, for each partition of the plurality, a number of image gradients associated with the partition using the image data of the partition, the image gradient representation of the digital image identifying each image gradient associated with at least one partition of the digital image.

4. The method of claim 3 , building the vocabulary of words further comprising:
quantizing the image gradients using k-means clustering, where k corresponds to a number of words in the vocabulary of words, and each of the quantized image gradients corresponds to a word in the vocabulary of words.

5. The method of claim 1 , determination of the frequency of occurrence further comprising:
determining, using the image gradient representation of a digital image of the plurality, the frequency of occurrence of each image gradient associated with each partition of the plurality of partitions of the digital image, the determining comprising determining a count of the number of occurrences of each image gradient associated with the digital image, each image gradient corresponding to a word in the vocabulary of words.

6. The method of claim 1 , further comprising:
representing, by the computing device, each digital image of the plurality of digital images as an unordered set of words and a frequency of occurrence of each word in the unordered set of words, each word in the unordered set corresponding to an image gradient of the plurality of image gradients representing the digital image.

7. The method of claim 6 , in the unordered set of words, a location of a partition in a digital image is ignored.

8. The method of claim 1 , further comprising:
representing, by the computing device, a digital image of the plurality as a histogram identifying a frequency of occurrence of each word associated with digital image using the image gradient representation of the digital image.

9. The method of claim 8 , a rectangle of the histogram representing a word of the vocabulary of words and a frequency of occurrence of the word associated with the digital image.

10. The method of claim 8 , differing histograms are indicative of differences in the digital images associated with the differing histograms.

11. The method of claim 1 , further comprising:
receiving, at the computing device, input indicative of a selection of the user of a representative digital image of the number; and
retrieving, via the computing device, a number of digital images from the cluster of the plurality being represented by the user-selected representative digital image; and
communicating, via the computing device, the number of digital images to the user for display on the device of the user.

12. The method of claim 1 , generation of the set of words for the received digital image further comprising:
partitioning the received digital image into a plurality of partitions;
identifying, for each partition of the plurality, a number of image gradients associated with the partition using the image data of the partition; and
generating the set of words for the received digital image using the number of image gradients associated with each partition of the received digital image.

13. A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions, that when executed by a processor associated with a computing device, performs a method comprising:
determining an image gradient representation of each digital image of a plurality of digital images, the image gradient representation identifying a plurality of image gradients determined for the digital image;
building a vocabulary of words using the image gradient representation of each digital image of the plurality of digital images;
determining, for each digital image of the plurality, a frequency of occurrence of each word, of the vocabulary of words, using the image gradient representation of the digital image;
generating, using the vocabulary, a probabilistic topic model comprising image-specific parameters for each digital image in the plurality of digital images and cluster-specific parameters for each cluster of a plurality of clusters;
assigning each digital image from the plurality of digital images to a cluster of the plurality of clusters using the digital image's image-specific parameters, the frequency of occurrence of each word associated with the digital image and the probabilistic topic model, each cluster corresponding to a word in the vocabulary;
identifying, for each cluster of the plurality of clusters, a representative digital image for the cluster;
receiving a digital image search request of a user;
generating a set of words, from the vocabulary of words, for the received digital image;
identifying, using the set of words generated for the received image as a query, a number of representative digital images, each representative digital image of the number being representative of a cluster, of the plurality of clusters, associated with at least one word of the set of words generated for the received image; and
communicating the number of representative digital i

Assignees

• Oath Inc

Inventors

• Li Jia
• Golbandi Nadav
• Zhang Xianxing

Classifications

• G06F18/23213

  with fixed number of clusters, e.g. K-means clustering · CPC title

• G06F18/2321

  using statistics or function optimisation, e.g. modelling of probability density functions · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06F16/5854

  using shape and object relationship · CPC title

• G06F17/30687

  Physics · mapped topic