Illumination estimation using natural scene statistics

Patent metadata

Field	Value
Publication number	US-9519839-B2
Application number	US-201414188670-A
Country	US
Kind code	B2
Filing date	Feb 24, 2014
Priority date	Feb 25, 2013
Publication date	Dec 13, 2016
Grant date	Dec 13, 2016

Abstract

A method for estimating illumination of an image captured by a digital system is provided that includes computing a feature vector for the image, identifying at least one best reference illumination class for the image from a plurality of predetermined reference illumination classes using the feature vector, an illumination classifier, and predetermined classification parameters corresponding to each reference illumination class, and computing information for further processing of the image based on the at least one best reference illumination class, wherein the information is at least one selected from a group consisting of color temperature and white balance gains.

First claim

What is claimed is:

1. A method for estimating illumination of an image captured by a digital system, the method comprising:
computing, with the digital system, ground truth white balance gains for each training image of a plurality of training images;
clustering, with the digital system, the training images into reference illumination classes based on the ground truth white balance gains;
deriving, with the digital system, feature vectors for each training image in each reference illumination class;
training, with the digital system, an illumination classifier based on the feature vectors, wherein classification parameters for each reference illumination class are determined;
computing, with the digital system, a feature vector for the image;
identifying, with the digital system, at least one best reference illumination class for the image from the reference illumination classes using the feature vector for the image, the illumination classifier, and the classification parameters corresponding to each reference illumination class;
computing, with the digital system, information for further processing of the image based on the at least one best reference illumination class, wherein the information is at least one selected from a group consisting of color temperature and white balance gains; and
performing, with the digital system, further processing on the image based on the information.

2. The method of claim 1 , wherein the illumination classifier is a multivariate Gaussian classifier (MVG), and wherein identifying at least one best reference illumination class comprises determining probabilities of observing the image in each of the reference illumination classes.

3. The method of claim 1 , wherein the illumination classifier is a multivariate Gaussian classifier (MVG), and wherein identifying at least one best reference illumination class comprises computing a Mahalanobis distance of the image to each of the plurality of reference illumination classes.

4. The method of claim 1 , wherein the at least one best reference illumination class comprises multiple best reference illumination classes, and wherein computing information for further processing comprises determining the color temperature for the image based on a weighted average of predetermined color temperatures of the multiple best reference illumination classes.

5. The method of claim 1 , wherein the at least one best reference illumination class comprises multiple best reference illumination classes, and wherein computing information for further processing comprises determining the white balance gains for the image as weighted averages of predetermined white balance gains of the multiple best reference illumination classes.

6. The method of claim 1 , wherein the plurality of training images comprises training images captured under multiple lighting conditions in multiple geographical locations, and wherein each training image has a corresponding reference image configured to provide ground truth color temperature and ground truth white balance gains for the training image.

7. A computer-implemented method for training an illumination classifier, the method comprising:
computing, with one or more processors, ground truth white balance gains for each training image of a plurality of training images;
clustering, with the one or more processors, the training images into reference illumination classes based on the ground truth white balance gains;
deriving, with the one or more processors, feature vectors for each training image in each reference illumination class;
training, with the one or more processors, an illumination classifier for the reference illumination classes based on the feature vectors, wherein classification parameters for each reference illumination class are determined; and
performing illumination estimation on an image based on the illumination classifier.

8. The method of claim 7 , wherein the plurality of training images comprises training images captured under multiple lighting conditions in multiple geographical locations, and wherein each training image has a corresponding reference image configured to provide ground truth color temperature and the ground truth white balance gains for the training image.

9. The method of claim 8 , further comprising:
computing a color temperature for each reference illumination class as a mean of the ground truth color temperatures of reference images corresponding to the training images in the reference illumination class; and
computing white balance gains for each reference illumination class as means of the ground truth white balance gains computed for each training image in the reference illumination class.

10. The method of claim 7 , wherein the illumination classifier is a multivariate Gaussian classifier (MVG) and the classification parameters of a reference illumination class comprise a mean vector and a covariance matrix of the feature vectors of the training images in the reference illumination class.

11. The method of claim 10 , wherein training an illumination classifier includes generating a feature matrix for each reference illumination class by arranging the feature vectors of the training images in the reference illumination class as columns of the feature matrix.

12. The method of claim 7 , wherein deriving feature vectors comprises computing, for each training image, a two-dimensional (2D) chromaticity histogram, reducing influence of dominant object color in the 2D chromaticity histogram, and transforming the 2D chromaticity histogram into a one-dimensional (1D) vector.

13. An apparatus configured to estimate illumination of an image, the apparatus comprising:
means for computing ground truth white balance gains for each training image of a plurality of training images;
means for clustering the training images into reference illumination classes based on the ground truth white balance gains;
means for deriving feature vectors for each training image in each reference illumination class;
means for training an illumination classifier based on the feature vectors, wherein classification parameters for each reference illumination class are determined;
means for capturing the image;
means for computing a feature vector for the image;
means for identifying at least one best reference illumination class for the image from the reference illumination classes using the feature vector for the image, the illumination classifier, and the classification parameters corresponding to each reference illumination class;
means for computing information for further processing of the image based on the at least one best reference illumination class, wherein the information is at least one selected from a group consisting of color temperature and white balance gains; and
means for performing further processing on the image based on the information.

14. The apparatus of claim 13 , wherein the illumination classifier is a multivariate Gaussian classifier (MVG), and wherein the means for identifying at least one best reference illumination class determines probabilities of observing the image in each of the plurality of reference illumination classes.

15. The apparatus of claim 13 , wherein the illumination classifier is a multivariate Gaussian classifier (MVG), and wherein the means for identifying at least one best reference illumination class computes a Mahalanobis distance of the image to each of the plurality of reference illumination classes.

16. The apparatus of claim 13 , wherein the at least one best reference illumination class comprises multiple best referen

Assignees

• Texas Instruments Inc

Inventors

• Zhang Buyue
• Batur Aziz Umit

Classifications

• G06V10/60 Primary

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• H04N23/88

  for colour balance, e.g. white-balance circuits or colour temperature control · CPC title

• H04N9/73

  Colour balance circuits, e.g. white balance circuits or colour temperature control (camera processing pipelines for colour balance H04N23/88) · CPC title

• G06K9/4661 Primary

  Physics · mapped topic

• G06T5/007

  Physics · mapped topic