Systems and methods for local tone mapping

Patent metadata

Field	Value
Publication number	US-9741099-B2
Application number	US-201514822316-A
Country	US
Kind code	B2
Filing date	Aug 10, 2015
Priority date	May 31, 2012
Publication date	Aug 22, 2017
Grant date	Aug 22, 2017

Abstract

Systems and methods for local tone mapping are provided. In one example, an electronic device includes an electronic display, an imaging device, and an image signal processor. The electronic display may display images of a first bit depth, and the imaging device may include an image sensor that obtains image data of a higher bit depth than the first bit depth. The image signal processor may process the image data, and may include local tone mapping logic that may apply a spatially varying local tone curve to a pixel of the image data to preserve local contrast when displayed on the display. The local tone mapping logic may smooth the local tone curve applied to the intensity difference between the pixel and another nearby pixel exceeds a threshold.

First claim

What is claimed is:

1. An electronic device comprising:
an electronic display configured to display images of a first bit depth;
an imaging device comprising an image sensor configured to obtain image data of a higher bit depth than the first bit depth; and
an image signal processor configured to process the image data, wherein the image signal processor comprises:
local tone mapping logic configured to apply a spatially varying local tone curve to a pixel of the image data to preserve local contrast when displayed on the display, wherein the local tone mapping logic is configured to smooth the local tone curve applied to the pixel unless an intensity difference between the pixel and one or more other nearby pixels exceeds a threshold indicative of an edge within the image data, wherein the local tone mapping logic is configured to apply the spatially varying local tone curve to the pixel by computing a gain based at least in part on an input luminance associated with the pixel and an output luminance obtained by a spatially varying lookup table.

2. The electronic device of claim 1 , wherein the local tone mapping logic is configured to compute the gain as the output luminance divided by the input luminance.

3. The electronic device of claim 1 , wherein the local tone mapping logic is configured to compute the gain as a smoothed value of the output luminance divided by the input luminance unless the intensity difference between the pixel and the one or more other nearby pixels exceeds the threshold.

4. The electronic device of claim 1 , wherein the local tone mapping logic comprises a bilateral filter configured to vary the gain depending on whether the intensity difference between the pixel and the one or more other nearby pixels exceeds the threshold.

5. The electronic device of claim 4 , wherein the bilateral filter comprises a horizontal bilateral filter.

6. An electronic device comprising:
an imaging device configured to obtain image data; and
an image signal processor configured to process the image data, wherein the image signal processor comprises:
local tone mapping logic configured to apply a spatially varying local tone curve to a pixel of the image data to preserve local contrast when displayed on the display, wherein the local tone mapping logic is configured to smooth the local tone curve applied to the pixel unless an intensity difference between the pixel and one or more other nearby pixels exceeds a threshold indicative of an edge within the image data, wherein the local tone mapping logic is configured to apply the spatially varying local tone curve to the pixel by computing a gain based at least in part on an input luminance associated with the pixel and an output luminance obtained by a spatially varying lookup table.

7. The electronic device of claim 6 , wherein the image signal processor comprises white pin logic configured to blend white into substantially saturated image data when the substantially saturated image data would otherwise appear gray, wherein the white pin logic comprises compensation gain logic to apply a spatially varying compensation gain value to either a maximum or minimum color component value, or both, of a pixel of the image data to determine when the pixel of the image data is substantially saturated but would otherwise appear gray.

8. The electronic device of claim 7 , wherein the compensation gain logic is configured to determine the spatially varying compensation gain value using a spatially varying compensation gain lookup table, wherein the spatially varying compensation gain lookup table comprises compensation gains that decrease radially from an optical center of the image data.

9. The electronic device of claim 8 , wherein the spatially varying compensation gain lookup table is configured to be indexed by a maximum or minimum color component value of an input pixel.

10. The electronic device of claim 7 , wherein the white pin logic is a component of an RGB-format image processing pipeline.

11. The electronic device of claim 6 , wherein the imaging device comprises a digital camera integrated with the electronic device, an external digital camera coupled to the electronic device via an input/output port, or some combination thereof.

12. The electronic device of claim 6 , comprising at least one of a desktop computer, a laptop computer, a tablet computer, a mobile cellular telephone, a portable media player, or any combination thereof.

13. An image signal processor configured to process image data acquired by an image sensor, wherein the image signal processor comprises:
means for implementing local tone mapping logic configured to:
apply a spatially varying local tone curve to a pixel of the image data to preserve local contrast when displayed on the display; and
smooth the local tone curve applied to the pixel unless an intensity difference between the pixel and one or more other nearby pixels exceeds a threshold indicative of an edge within the image data, wherein the local tone mapping logic is configured to apply the spatially varying local tone curve to the pixel by computing a gain based at least in part on an input luminance associated with the pixel and an output luminance obtained by a spatially varying lookup table.

14. The image signal processor of claim 13 , comprising means for implementing white pin logic configured to blend white into substantially saturated image data when the substantially saturated image data would otherwise appear gray.

15. The image signal processor of claim 14 , wherein the white pin logic comprises means for compensation gain logic to apply a spatially varying compensation gain value to either a maximum or minimum color component value, or both, of a pixel of the image data to determine when the pixel of the image data is substantially saturated but would otherwise appear gray.

16. The image signal processor of claim 15 , wherein the compensation gain logic is configured to determine the spatially varying compensation gain value using a spatially varying compensation gain lookup table, wherein the spatially varying compensation gain lookup table comprises compensation gains that decrease radially from an optical center of the image data.

17. The image signal processor of claim 14 , comprising means for highlight recovery logic configured to recover image information from clipped or nearly clipped pixels of the image data before reaching the white pin logic.

18. The electronic device of claim 1 , wherein the image signal processor comprises white pin logic configured to blend white into substantially saturated image data when the substantially saturated image data would otherwise appear gray.

Assignees

• Apple Inc

Inventors

• Lim Suk Hwan
• Silverstein D Amnon
• Cote Guy
• Hordley Steven David
• Finlayson Graham
• Ku Weichun
• Bratt Joseph P

Classifications

• G06T2207/20012

  Locally adaptive · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T5/001 Primary

  Physics · mapped topic

• G06T2207/20028

  Bilateral filtering · CPC title