Refining image relevance models

Patent metadata

Field	Value
Publication number	US-9454600-B2
Application number	US-201213363979-A
Country	US
Kind code	B2
Filing date	Feb 1, 2012
Priority date	Sep 30, 2011
Publication date	Sep 27, 2016
Grant date	Sep 27, 2016

Abstract

Methods, systems and apparatus for refining image relevance models. In general, one aspect includes receiving a trained image relevance model that generates relevance measures of content feature values of images to a query, identifying a first threshold number of common content feature values for the set of training images, the common content feature values being identified as a set of content feature values that are each shared by at least a portion of the training images, identifying a subset of the set of training images having a quantity of the common content feature values greater than a second threshold number of content features values, and generating a re-trained image relevance model based on content feature values of the set of training images, wherein content feature values of the subset of training images are weighted higher than content feature values of the training images not in the subset.

First claim

What is claimed is:

1. A method comprising:
identifying a set of training images previously used to train an image relevance model that generates relevance measures of images to a query based on content feature values of the images, the query being a unique set of one or more query terms; identifying typical visual features for the set of training images, including:
for two or more visual features:
determining that at least a given portion of the training images from the set of training images that were used to train the image relevance model each include the visual features, and
in response to determining that at least a given portion of the training images each include the visual features, identifying the visual features as typical visual features for the set of training images;
identifying, as a subset of training images from the set of training images, training images having at least a threshold portion of the identified typical visual features, wherein the subset does not include all images within the set of training images;
assigning, to each training image in the subset of the training images, a weight that increases in proportion to the number of the identified typical visual features that the training image has, wherein each training image in the subset has a higher weight than weights assigned to other images from the set of training images that are not in the subset of training images; and
generating a re-trained image relevance model based on visual features of the set of training images and the weights assigned to the training images in the subset of training images that have at least the threshold portion of the identified typical visual features.

2. The method of claim 1 , further comprising assigning a boost value to each image in the subset of the set of training images, the boost value based on a score received from a low-capacity image relevance model trained using the set of training images.

3. The method of claim 1 , further comprising:
training a low-capacity image relevance model based on visual features of the images, the low-capacity image relevance model representing a fixed number of the visual features for the set of training images.

4. The method of claim 3 , wherein identifying, as a subset of the set of training images, training images having at least a threshold portion of the identified typical visual features comprises:
receiving a score for each image in the set of training images from the low-capacity image relevance model, and identifying images having a score greater than a score threshold as included in the subset of the set of training images.

5. The method of claim 1 , further comprising assigning a multiplier value to each image in the subset of training images, the multiplier value being greater than one.

6. The method of claim 5 , wherein generating a re-trained image relevance model comprises generating the re-trained image relevance model based on the visual features for the subset of training images and the assigned multiplier value.

7. A system, comprising:
a data processing apparatus; and
a memory coupled to the data processing apparatus having instructions stored thereon which, when executed by the data processing apparatus cause the data processing apparatus to perform operations comprising:
identifying a set of training images previously used to train an image relevance model that generates relevance measures of images to a query based on content feature values of the images, the query being a unique set of one or more query terms;
identifying typical visual features for the set of training images, including:
for two or more visual features:
determining that at least a given portion of the training images from the set of training images that were used to train the image relevance model each include the visual features, and
in response to determining that at least a given portion of the training images each include the visual features, identifying the visual features as typical visual features for the set of training images;
identifying, as a subset of training images from the set of training images, training images having at least a threshold portion of the identified typical visual features, wherein the subset does not include all images within the set of training images;
assigning, to each training image in the subset of the training images, a weight that increases in proportion to the number of the identified typical visual features that the training image has, wherein each training image in the subset has a higher weight than weights assigned to other images from the set of training images that are not in the subset of training images; and
generating a re-trained image relevance model based on visual features of the set of training images and the weights assigned to the training images in the subset of training images that have at least the threshold portion of the identified typical visual features.

8. The system of claim 7 , the operations further comprising assigning a boost value to each image in the subset of the set of training images, the boost value based on a score received from a low-capacity image relevance model trained using the set of training images.

9. The system of claim 7 , the operations further comprising:
training a low-capacity image relevance model based on visual features of the images, the low-capacity image relevance model representing a fixed number of the visual features for the set of training images.

10. The system of claim 9 , wherein identifying, as a subset of the set of training images, training images having at least a threshold portion of the identified typical visual features comprises:
receiving a score for each image in the set of training images from the low-capacity image relevance model, and identifying images having a score greater than a score threshold as included in the subset of the set of training images.

11. The system of claim 7 , the operations further comprising assigning a multiplier value to each image in the subset of training images, the multiplier value being greater than one.

12. The system of claim 11 , wherein generating a re-trained image relevance model comprises generating the re-trained image relevance model based on the visual features for the subset of training images and the assigned multiplier value.

13. Non-transitory computer readable media storing software comprising instructions executable by a processing device and upon such execution cause the processing device to perform operations comprising:
identifying a set of training images previously used to train an image relevance model that generates relevance measures of images to a query based on content feature values of the images, the query being a unique set of one or more query terms;
identifying typical visual features for the set of training images, including:
for two or more visual features:
determining that at least a given portion of the training images from the set of training images that were used to train the image relevance model each include the visual features, and
in response to determining that at least a given portion of the training images each include the visual features, identifying the visual features as typical visual features for the set of training images;
identifying, as a subset of training images from the set of training images, training images having at least a threshold portion of the identified typical visual features, wherein the subset does not include all images within the set of training images;
assigning, to each training image in the subset of the training images, a weight that increases in proportion to the number of

Assignees

• Duerig Thomas J
• Weston Jason E
• Rosenberg Charles J
• Gu Kunlong
• Bengio Samy
• Google Inc

Inventors

• Duerig Thomas J
• Weston Jason E
• Rosenberg Charles J
• Gu Kunlong
• Bengio Samy

Classifications

• G06V30/19147

  Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/29

  Graphical models, e.g. Bayesian networks · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V30/194

  References adjustable by an adaptive method, e.g. learning · CPC title