System and method for generating training cases for image classification

Patent metadata

Field	Value
Publication number	US-9251437-B2
Application number	US-201313970869-A
Country	US
Kind code	B2
Filing date	Aug 20, 2013
Priority date	Dec 24, 2012
Publication date	Feb 2, 2016
Grant date	Feb 2, 2016

Abstract

A system and method for generating training images. An existing training image is associated with a classification. The system includes an image processing module that performs color-space deformation on each pixel of the existing training image and then associates the classification to the color-space deformed training image. The technique may be applied to increase the size of a training set for training a neural network.

First claim

The invention claimed is:

1. A method performed by one or more computers, the method comprising:
obtaining training data for a neural network, wherein the training data comprises a plurality of base training images and respective classification data for each of the base training images, and wherein the neural network is configured to receive an input image and predict classification data for the input image, wherein each image comprises data representing pixels having a respective color;
generating one or more color-deformed images from the base training images of the training data, the generating comprising, for each of the plurality of base training images:
performing a principal component analysis (PCA) on pixels in a first region of the base training image to obtain a plurality of eigenvector-eigenvalue pairs of a covariance matrix of red green blue (RGB) pixel values from the pixels in the first region of the base training image; and
applying an intensity transformation of pixel colors of the pixels in the first region of the base training image, comprising:
randomly selecting a respective value for each eigenvector-eigenvalue pair of the covariance matrix; and
for each pixel in the first region of the base training image, applying a transformation to the pixel colors of the pixel based on the eigenvector-eigenvalue pairs and the randomly-selected values; and

adding the one or more color-deformed images to the training data for the neural network.

2. The method of claim 1 , wherein the classification data for each of the base training images comprises data that labels one more objects in the base training image.

3. The method of claim 1 , wherein each color-deformed image is generated from a respective base training image, and generating each color-deformed image comprises applying one or more color-space deformations to pixel colors of the respective base image.

4. The method of claim 1 , wherein generating the one or more color-deformed images from the plurality of base training images of the training data further comprises:
generating a respective color-deformed image from each of the plurality of base training images, comprising applying the intensity transformation to pixel colors of the pixels in the set of pixels in the first region of the respective base training image, and wherein the method further comprises:
associating each color-deformed image with the classification data for the base training image from which the color-deformed image was generated; and
adding each color-deformed image and the associated classification data to the set of training data.

5. The method of claim 4 , wherein applying the one or more respective color-space deformations to pixel colors of the first base training image comprises:
applying a second color intensity transformation to pixel colors of pixels in a second, different region of the base training image.

6. The method of claim 1 , wherein the pixel of the first base training image is an RGB image pixel represented by I xy =[I xy R ,I xy G ,I xy B ] T , wherein applying the transformation comprises adding [p 1 ,p 2 ,p 3 ][α 1 √λ 1 ,α 2 √λ 2 ,α 3 √λ 3 ] T to I xy , and wherein p i is an i-th eigenvector of the covariance matrix, λ i is an i-th eigenvalue of the covariance matrix, and α i is a randomly-selected value.

7. A system comprising one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
obtaining training data for a neural network, wherein the training data comprises a plurality of base training images and respective classification data for each of the base training images, and wherein the neural network is configured to receive an input image and predict classification data for the input image, wherein each image comprises data representing pixels having a respective color;
generating one or more color-deformed images from the base training images of the training data, the generating comprising, for each of the plurality of base training images:
performing a principal component analysis (PCA) on pixels in a first region of the base training image to obtain a plurality of eigenvector-eigenvalue pairs of a covariance matrix of red green blue (RGB) pixel values from the pixels in the first region of the base training image; and
applying an intensity transformation to pixel colors of the pixels in the first region of the base training image, comprising:
randomly selecting a respective value for each eigenvector-eigenvalue pair of the covariance matrix; and
for each pixel in the first region of the base training image, applying a transformation to the pixel colors of the pixel bsed on the eigenvector-eigenvalue pairs and the randomly-selected values; and

adding the one or more color-deformed images to the training data for the neural network.

8. The system of claim 7 , wherein the classification data for each of the base training images comprises data that labels one more objects in the base training image.

9. The system of claim 7 , wherein each color-deformed image is generated from a respective base training image, and generating each color-deformed image comprises applying one or more color-space deformations to pixel colors of the respective base image.

10. The system of claim 7 , wherein generating the one or more color-deformed images from the plurality of base training images of the training data further comprises:
generating a respective color-deformed image from each of the plurality of base training images, comprising applying the intensity transformation to pixel colors of the pixels in the set of pixels in the first region of the respective base training image, and wherein the method further comprises:
associating each color-deformed image with the classification data for the base training image from which the color-deformed image was generated; and
adding each color-deformed image and the associated classification data to the set of training data.

11. The system of claim 10 , wherein generating the respective color-deformed image from each of the plurality of base training images further comprises:
applying a different, second color intensity transformation to pixel colors of pixels in a second, different region of the base training image.

12. The system of claim 7 , wherein the pixel of the base training image is an RGB image pixel represented by I xy =[I xy R ,I xy G ,I xy B ] T , wherein applying the transformation comprises adding [p 1 ,p 2 ,p 3 ][α 1 √λ 1 ,α 2 ,√λ 2 ,α 3 29 λ 3 ] T to I xy , and wherein p i is an i-th eigenvector of the covariance matrix, λ i is an i-th eigenvalue of the covariance matrix, and α i is a randomly-selected value.

13. A non-transitory computer-readable storage medium encoded with a computer program, the program comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
obtaining training data for a neural network, wherein the training data comprises a plurality of base training images and respective classification data for each of the base training images, wherein the neural network is configured to receive an input image and predict classification data for the input image, and wherein each image comprises data representing pixels having a respective color;
generating one or more color-deformed images from the base training images of the training data, the generating comprising, for each of the plurality of base training images:
performing a principal component analysis (PCA) on pixels in a first region

Assignees

• Google Inc

Inventors

• Krizhevsky Alexander
• Sutskever Ilya
• Hinton Geoffrey E

Classifications

• G06F18/28 Primary

  Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries · CPC title

• G06K9/6255 Primary

  Physics · mapped topic