Per-pixel photometric contrast enhancement with noise control

Patent metadata

Field	Value
Publication number	US-11010873-B2
Application number	US-202015930250-A
Country	US
Kind code	B2
Filing date	May 12, 2020
Priority date	Aug 10, 2018
Publication date	May 18, 2021
Grant date	May 18, 2021

Abstract

Embodiments relate to enhancing local contrast in an image. A bilateral high pass filter generates a high frequency value for each pixel of an input image, based on a convolution using photometric kernel coefficients associated with other pixels around the pixel. A noise control circuit generates a modulated high frequency value for the pixel based on a noise model for the image defining a noise threshold value for modifying the high frequency value. The modulated high frequency value for the pixel is then combined with a pixel value of the pixel to generate an enhanced value for the pixel. Enhanced values for pixels of the image may be generated to provide the local contrast enhancement for the input image.

First claim

What is claimed is:

1. An apparatus for enhancing an image, comprising:
a bilateral high pass filter configured to generate a high frequency value for each of pixels in the image by performing convolution using photometric kernel coefficients and photometric differences between a pixel value of each pixel and pixel values of other pixels in a patch of the image including each pixel, each photometric kernel coefficient representing a measure of photometric similarity between each pixel and a corresponding pixel in the patch, the high frequency value corresponding to a level of photometric contrast between each pixel and the other pixels; and
a circuit configured to generate an enhanced value for each pixel by combining a modulated version of the high frequency value for each pixel with the pixel value of each pixel, the modulated version generated using a noise threshold value.

2. The apparatus of claim 1 , further comprising a photometric processor configured to:
determine each of the photometric kernel coefficients using a photometric distance between the pixel value of each pixel and another pixel value for the corresponding pixel in the patch.

3. The apparatus of claim 1 , wherein the bilateral high pass filter is further configured to generate the high frequency value by performing bilateral high pass filtering of a single color component of each pixel and the other pixels in the patch.

4. The apparatus of claim 1 , further comprising a noise control circuit configured to:
attenuate the high frequency value when the high frequency value is below the noise threshold value to generate the modulated version; and
amplify the high frequency value when the high frequency value exceeds the noise threshold value to generate the modulated version.

5. The apparatus of claim 1 , further comprising a noise control circuit configured to:
determine the noise threshold value based on a noise model and at least one color component of one or more pixel values of the image;
determine a modulation signal based at least in part on the high frequency value and the noise threshold value; and
multiply the high frequency value with the modulation signal to obtain the modulated version of the high frequency value.

6. The apparatus of claim 5 , wherein:
the modulation signal increases as the high frequency value increases until the high frequency value reaches the noise threshold value; and
the modulation signal is equal to a defined maximum value for the modulation signal when the high frequency value exceeds the noise threshold value.

7. The apparatus of claim 5 , wherein the noise control circuit is further configured to store the noise model as a look-up table representing threshold frequency values as a function of pixel values of the image for one or more color components associated with the pixel values.

8. The apparatus of claim 1 , further comprising a radial gain calculator circuit configured to:
determine a radial gain based on information about a position of each pixel relative to a central axis of the image; and
multiply the modulated version of the high frequency value with the radial gain to generate an adjusted modulated high frequency value for each pixel.

9. The apparatus of claim 8 , wherein the circuit is further configured to add the adjusted modulated high frequency value for each pixel to a single color component of the pixel value of each pixel to generate the enhanced value for each pixel.

10. A method for enhancing an image, the method comprising:
generating a high frequency value for each of pixels in the image by performing convolution using photometric kernel coefficients and photometric differences between a pixel value of each pixel and pixel values of other pixels in a patch of the image including each pixel, each photometric kernel coefficient representing a measure of photometric similarity between each pixel and a corresponding pixel in the patch, the high frequency value corresponding to a level of photometric contrast between each pixel and the other pixels; and
generating an enhanced value for each pixel by combining a modulated version of the high frequency value for each pixel with the pixel value of each pixel, the modulated version generated using a noise threshold value.

11. The method of claim 10 , further comprising:
determining each of the photometric kernel coefficients using a photometric distance between the pixel value of each pixel and another pixel value for the corresponding pixel in the patch.

12. The method of claim 10 , further comprising:
generating the high frequency value by performing bilateral high pass filtering of a single color component of each pixel and the other pixels in the patch.

13. The method of claim 10 , further comprising:
attenuating the high frequency value when the high frequency value is below the noise threshold value to generate the modulated version; and
amplifying the high frequency value when the high frequency value exceeds the noise threshold value to generate the modulated version.

14. The method of claim 10 , further comprising:
determining the noise threshold value based on a noise model and at least one color component of one or more pixel values of the image;
determining a modulation signal based at least in part on the high frequency value and the noise threshold value; and
multiplying the high frequency value with the modulation signal to obtain the modulated version of the high frequency value.

15. The method of claim 14 , wherein:
the modulation signal increases as the high frequency value increases until the high frequency value reaches the noise threshold value; and
the modulation signal is equal to a defined maximum value for the modulation signal when the high frequency value exceeds the noise threshold value.

16. The method of claim 10 , further comprising:
determining a radial gain based on information about a position of each pixel relative to a central axis of the image;
multiplying the modulated version of the high frequency value with the radial gain to generate an adjusted modulated high frequency value for each pixel; and
adding the adjusted modulated high frequency value for each pixel to a single color component of the pixel value of each pixel to generate the enhanced value for each pixel.

17. A system, comprising:
an image sensor configured to obtain an image having a plurality of color components;
an image signal processor coupled to the image sensor, the image signal processor configured to perform de-noising of the image to obtain a de-noised version of the image having one or more color components of the plurality of color components, the image signal processor including:
a bilateral high pass filter configured to generate a high frequency value for each of pixels in the de-noised version of the image by performing convolution using photometric kernel coefficients and photometric differences between a pixel value of each pixel and pixel values of other pixels in a patch of the image including each pixel, each photometric kernel coefficient representing a measure of photometric similarity between each pixel and a corresponding pixel in the patch, the high frequency value corresponding to a level of photometric contrast between each pixel and the other pixels, and
a circuit configured to generate an enhanced value for each pixel by combining a modulated version of the high frequency value for each pixel with the pixel value of each pixel, the modulated version generated using a noise threshold value.

18. The system of claim

Assignees

• Apple Inc

Inventors

• Smirnov Maxim

Classifications

• G06T5/20

  using local operators · CPC title

• G06T3/40

  Scaling of whole images or parts thereof, e.g. expanding or contracting · CPC title

• G06T5/002 Primary

  Physics · mapped topic

• G06T5/008

  Physics · mapped topic

• G06T5/003

  Physics · mapped topic