Training image sampling

Patent metadata

Field	Value
Publication number	US-9230194-B2
Application number	US-201314027512-A
Country	US
Kind code	B2
Filing date	Sep 16, 2013
Priority date	Sep 16, 2013
Publication date	Jan 5, 2016
Grant date	Jan 5, 2016

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for selecting training images. One of the methods includes determining, for each of a plurality of labels that each designate a respective food class of a plurality of food classes, a respective measure of importance. A respective sample size is determined for the label based on the respective measure of importance of the label. A number of training images are selected for each respective label according to the determined sample size for the label. A predictive model is trained using the selected training images as training data.

First claim

What is claimed is:

1. A computer-implemented method of classifying an image into a food class comprising:
for each of a plurality of labels that each designate a respective food class of a plurality of food classes, wherein each food class represents a different food item:
determining a respective measure of importance of the label of the plurality of labels, and
determining a respective sample size for the label of the plurality of labels, wherein the sample size is based on the respective measure of importance of the label;

determining, for each label of a subset of labels having smallest respective measures of importance, that a collection of labeled images includes fewer images having the label than a respective determined sample size for the label;
selecting, from the collection of labeled images, for each respective label of the plurality of labels, a number of training images according to the determined sample size for the label, including selecting, for each label of the subset of labels having the smallest respective measures of importance, multiple instances of at least one image having the label; and
training a predictive model using the selected training images as training data.

2. The method of claim 1 , further comprising:
boosting a measure of importance for a first label of the subset of labels having the smallest respective measures of importance;
determining an updated sample size based on the boosted measure of importance for the first label; and
selecting a number of training images for the first label based on the updated sample size for the first label, wherein at least one image having the first label is selected multiple times as a training image for the first label.

3. The method of claim 2 , further comprising:
determining that the collection of labeled images includes a number of images having a second label that is larger than the sample size assigned to a corresponding second label; and
selecting a number of training images for the second label based on the sample size for the second label, wherein each image in the number of training images for the second label is used no more than once as a training image for the second label.

4. The method of claim 1 , wherein training the predictive model comprises:
selecting, for a subsequent training iteration, a same number of training images for a first label, wherein at least one image in the number of training images for the first label is used multiple times as a training image for the first label.

5. The method of claim 1 , further comprising:
assigning a same measure of importance to (i) a first label for a first food class having a first number of images in the collection of images, and (ii) a second label for a second food class having a smaller second number of images in the collection of images.

6. The method of claim 5 , wherein at least one image having the second label is used multiple times as a training image for the second label.

7. The method of claim 1 , further comprising:
receiving an image query that specifies an image of a food item;
determining a label for the food item using the trained model;
generating a search query based on the determined label;
providing the search query to a search engine; and
providing a result from the search engine in response to the image query.

8. The method of claim 7 , wherein the search engine is a general Internet search engine, a nutritional information search engine, or a recipe search engine.

9. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
for each of a plurality of labels that each designate a respective food class of a plurality of food classes, wherein each food class represents a different food item:
determining a respective measure of importance of the label of the plurality of labels, and
determining a respective sample size for the label of the plurality of labels, wherein the sample size is based on the respective measure of importance of the label;

determining, for each label of a subset of labels having smallest respective measures of importance, that a collection of labeled images includes fewer images having the label than a respective determined sample size for the label;
selecting, from the collection of labeled images, for each respective label of the plurality of labels, a number of training images according to the determined sample size for the label including selecting, for each label of the subset of labels having the smallest respective measures of importance, multiple instances of at least one image having the label; and
training a predictive model using the selected training images as training data.

10. The system of claim 9 , wherein the operations further comprise:
boosting a measure of importance for a first label of the subset of labels having the smallest respective measures of importance;
determining an updated sample size based on the boosted measure of importance for the first label; and
selecting a number of training images for the first label based on the updated sample size for the first label, wherein at least one image having the first label is selected multiple times as a training image for the first label.

11. The system of claim 10 , wherein the operations further comprise:
determining that the collection of labeled images includes a number of images having a second label that is larger than the sample size assigned to a corresponding second label; and
selecting a number of training images for the second label based on the sample size for the second label, wherein each image in the number of training images for the second label is used no more than once as a training image for the second label.

12. The system of claim 9 , wherein training the predictive model comprises:
selecting, for a subsequent training iteration, a same number of training images for a first label, wherein at least one image in the number of training images for the first label is used multiple times as a training image for the first label.

13. The system of claim 9 , wherein the operations further comprise:
assigning a same measure of importance to (i) a first label for a first food class having a first number of images in the collection of images, and (ii) a second label for a second food class having a smaller second number of images in the collection of images.

14. The system of claim 13 , wherein at least one image having the second label is used multiple times as a training image for the second label.

15. The system of claim 9 , wherein the operations further comprise:
receiving an image query that specifies an image of a food item;
determining a label for the food item using the trained model;
generating a search query based on the determined label;
providing the search query to a search engine; and
providing a result from the search engine in response to the image query.

16. The system of claim 15 , wherein the search engine is a general Internet search engine, a nutritional information search engine, or a recipe search engine.

17. A computer program product, encoded on one or more non-transitory computer storage media, comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
for each of a plurality of labels that each designate a respective food class of a plurality of food classes, wherein each food class represents a dif

Assignees

• Google Inc

Inventors

• Rabinovich Andrew
• Adam Hartwig

Classifications

• G06V30/19147 Primary

  Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V30/194

  References adjustable by an adaptive method, e.g. learning · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06K9/66 Primary

  Physics · mapped topic