Per pixel color correction filtering

Patent metadata

Field	Value
Publication number	US-9979942-B2
Application number	US-201615198552-A
Country	US
Kind code	B2
Filing date	Jun 30, 2016
Priority date	Jun 30, 2016
Publication date	May 22, 2018
Grant date	May 22, 2018

Abstract

Embodiments relate to color correction circuit operations performed by an image signal processor. The color correction circuit computes optimal color correction matrix on a per-pixel basis and adjusts it based on relative noise standard deviations of the color channels to steer the matrix.

First claim

What is claimed is:

1. An image signal processor comprising:
a per-pixel color correction matrix circuit configured to compute a modified color correction matrix for each pixel in image data, and generate first color values by at least applying the modified color correction matrix to the image data, the per-pixel color correction matrix circuit comprising:
a standard color correction matrix circuit configured to receive a smoothed version of the image data with a plurality of color channels, the standard color correction matrix circuit further configured to apply the standard color correction matrix to the smoothed version of the image data to generate first color-corrected signals, wherein applying the modified color correction matrix to the smoothed version of the image data results in same color values as another smoothed version of the image data applied with a standard color matrix;
a normalizer circuit configured to normalize the first color-corrected signals to a noise variance value of pixels in the image data to generate normalized color-corrected signals, the noise variance value derived from a noise model of the pixels in the image data, wherein an expected noise variance value of the first color values is lower than another expected noise variance value in color values obtained by applying the standard color matrix to the image data; and
a scaler circuit configured to compute coefficients of the modified color correction matrix for the pixels in the image data, the scaler circuit further configured to compute first intermediate values as the first color values by applying a scale factor to the smoothed version of the image data, the scale factor based on the normalized color-corrected signals.

2. The image signal processor of claim 1 , where the per-pixel color correction matrix circuit further comprises:
a high frequency component circuit configured to compute second intermediate values corresponding to a difference between the image data and the smoothed version of the image data; and
a blender circuit configured to blend the first intermediate values and a fraction of the second intermediate values.

3. The image signal processor of claim 1 , further comprising a demosaicer circuit configured to:
produce the image data from raw image data in a Bayer pattern; and
produce the smoothed version of the image data by convolving spatial filter kernels with the raw image data.

4. The image signal processor of claim 3 , wherein the demosaicer circuit is configured to produce the smoothed version of the image data by selecting a spatial filter kernel from the spatial filter kernels for a Bayer pixel component of the raw image data and a data channel associated with the Bayer pixel component.

5. The image signal processor of claim 1 , wherein the smoothed version of the image data is a low-pass filtered version of the image data.

6. The image signal processor of claim 1 , wherein per-pixel color correction matrix circuit is further configured to:
retrieve an inverse standard deviation value from a noise model look-up table corresponding to the pixels of the image data, the noise model look-up table mapping signal intensity values of pixels to inverse standard deviation values; and
derive the noise variance value from the inverse standard deviation value retrieved from the noise model look-up table.

7. The image signal processor of claim 1 , wherein per-pixel color correction matrix circuit is further configured to:
retrieve a standard deviation value from a noise model look-up table corresponding to the pixels of the image data, the noise model look-up table mapping signal intensity values of pixels to standard deviation values; and
derive the noise variance value from the standard deviation value retrieved from the noise model look-up table.

8. The image signal processor of claim 1 , wherein the scaler circuit is configured to compute the coefficients of the modified color correction matrix based on the normalized color-corrected signals, the image data, and the smoothed version of the image data.

9. The image signal processor of claim 8 , wherein the scaler is configured to:
determine a scaled sum of image data signals based on the normalized color-corrected signals and the image data;
determine another scaled sum of a smoothed version of the image data signals based on the normalized color-corrected signals and a smoothed version of the image data signals;
determine a scale factor based on a ratio of the scaled sum of the image data signals and the other scaled sum of the smoothed version of the image data signals; and
apply the scale factor to the normalized color-corrected signals.

10. The image signal processor of claim 1 , further comprising a high frequency processor configured to:
determine a luminance high frequency data signal and a maximum high frequency data signal; and
determine a weight based on the luminance high frequency data signal and the maximum high frequency data signal.

11. The image signal processor of claim 10 , wherein a fraction of second intermediate values is determined by applying the weight to the second intermediate values, the method further comprising:
blending the first intermediate values and the fraction of the second intermediate values.

12. A method of performing color correction on image data with a plurality of color channels, comprising:
computing a modified color correction matrix for each pixel in the image data;
generating first color values by at least applying the modified color correction matrix to the image data;
applying a standard color correction matrix to a smoothed version of the image data to generate first color-corrected signals, wherein applying the modified color correction matrix to the smoothed version of the image data results in same color values as another smoothed version of the image data applied with a standard color matrix;
normalizing the first color-corrected signals to a noise variance value of a pixel in the image data to generate normalized color-corrected signals, the noise variance value derived from a noise model of pixels in the image data, wherein an expected noise variance value of the first color values is lower than another expected noise variance value in color values obtained by applying the standard color matrix to the image data; and
computing first intermediate values as the first color values by applying a scale factor to the smoothed version of the image data, the scale factor based on the normalized color-corrected signals.

13. The method of claim 12 , further comprising:
computing second intermediate values corresponding to a difference between the image data and the smoothed version of the image data; and
blending the first intermediate values and a fraction of the second intermediate values.

14. The method of claim 12 , further comprising:
producing the image data from a raw image data in a Bayer pattern; and
producing the smoothed version of the image data by convolving spatial filter kernels with the raw image data.

15. The method of claim 14 , wherein the smoothed version of the image data is produced by selecting a spatial filter kernel from the spatial filter kernels for a Bayer pixel component of the raw image data and a data channel associated with the Bayer pixel component.

16. The method of claim 12 , wherein the smoothed version of the image data is a low-pass filtered version of the image data.

17. The method of claim 12 , further comprising:
retrieving an inverse standard deviation value from a noise model look-up table correspondin

Assignees

• Apple Inc

Inventors

• Lim Suk Hwan
• Silverstein D Amnon
• Pope David R

Classifications

• H04N25/6153

  for colour signals · CPC title

• H04N9/76

  for mixing of colour signals (H04N9/75 takes precedence) · CPC title

• H04N9/04

  Electricity · mapped topic

• H04N9/646 Primary

  for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters · CPC title

• G06T3/4015

  Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns · CPC title