Global tone mapping

Patent metadata

Field	Value
Publication number	US-10530995-B2
Application number	US-201715821032-A
Country	US
Kind code	B2
Filing date	Nov 22, 2017
Priority date	Jun 12, 2015
Publication date	Jan 7, 2020
Grant date	Jan 7, 2020

Abstract

A system accesses an image with each pixel of the image having luminance values each representative of a color component of the pixel. The system generates a first histogram for aggregate luminance values of the image, and accesses a target histogram for the image representative of a desired global image contrast. The system computes a transfer function based on the first histogram and the target histogram such that when the transfer function is applied, a histogram of the modified aggregate luminance values is within a threshold similarity of the target histogram. The system modifies the image by applying the transfer function to the luminance values of the image to produce a tone mapped image, and outputs the modified image.

First claim

What is claimed is:

1. A system comprising:
an image sensor;
a memory; and
a processor, wherein the memory stores instructions executable by the processor to cause the system to:
access an image captured by the image sensor;
determine a first histogram for luminance values of the image;
access a target histogram for the image;
compute a transfer function based on the first histogram and the target histogram such that when the transfer function is applied to pixel values of the image to create a set of modified luminance values, a histogram of the modified luminance values is within a threshold similarity of the target histogram;
generate an exposure correction curve based on a lookup table, the exposure correction curve compensating for portions of the first histogram that are representative of an underexposure of the image within a corresponding portion of a color channel, and the lookup table producing a linear output for brightness values below a threshold and a compressed non-linear output for brightness values above a threshold;
apply the exposure correction curve to pixel values of the image to generate exposure corrected values of the image;
transpose the transfer function from a post-gamma curve space to a linear space of the image;
apply the transfer function to pixel values of the image to produce a tone mapped image; and
output the tone mapped image.

2. The system of claim 1 , wherein the memory stores instructions executable by the processor to cause the system to:
compute, for each pixel of the image, a luminance value based on a luminance function applied to a set of values corresponding to respective color components of the pixel.

3. The system of claim 1 , wherein the memory stores instructions executable by the processor to cause the system to:
generate a histogram for a color channel of the image; and
apply a blackpoint to the image for the color channel based on the generated histogram for the color channel to establish a lower bound for the color channel.

4. The system of claim 1 , wherein the memory stores instructions executable by the processor to cause the system to:
adjust the transfer function to remove above-threshold amplification of noise resulting from an application of the transfer function to the image.

5. The system of claim 4 , wherein the memory stores instructions executable by the processor to cause the system to:
compute a derivative of the transfer function;
adjust one of a high threshold area and a low threshold area of the derivative of the transfer function such that the high threshold area and the low threshold area are equal;
modify the derivative by removing the high threshold area and the low threshold area from the derivative; and
compute a modified transfer function based on the modified derivative.

6. The system of claim 1 , wherein the memory stores instructions executable by the processor to cause the system to:
apply an unsharp mask to portions of the image that include areas of interest in order to increase local contrast.

7. The system of claim 6 , wherein the memory stores instructions executable by the processor to cause the system to:
determine one or more changed image portions of the tone mapped image that are one of compressed and amplified compared to a corresponding image portion of the image; and
modify these changed image portions to increase the local contrast in these changed image portions.

8. The system of claim 1 , wherein the memory stores instructions executable by the processor to cause the system to:
in response to a detecting that the image is uniform, modify the transfer function to reduce a gain of the transfer function.

9. The system of claim 1 , wherein the memory stores instructions executable by the processor to cause the system to:
obtain scene classification information for the image; and
modify the transfer function based on the scene classification information.

10. The system of claim 9 , wherein the scene classification information is determined by an automatic white balance module configured to apply white balance correction to the image.

11. The system of claim 1 , wherein the memory stores instructions executable by the processor to cause the system to:
determine a blue sky indication that reflects a proportion of a scene depicted in the image that includes blue sky; and
modify the transfer function based on the blue sky indication to reduce a gain of the transfer function.

12. The system of claim 11 , wherein the blue sky indication is determined by an automatic white balance module configured to apply white balance correction to the image.

13. The system of claim 1 , wherein the memory stores instructions executable by the processor to cause the system to:
determine a faces indication that reflects a proportion of a scene depicted in the image that includes one or more human faces; and
modify the transfer function based on the faces indication to reduce a gain of the transfer function.

14. A method comprising:
accessing an image captured by an image sensor;
computing a first histogram for luminance values of the image;
accessing a target histogram for the image;
computing a transfer function based on the first histogram and the target histogram such that when the transfer function is applied to pixel values of the image to create a set of modified luminance values, a histogram of the modified luminance values is within a threshold similarity of the target histogram;
obtaining scene classification information for the image;
modifying the transfer function based on the scene classification information, wherein modifying the transfer function comprises multiplying a difference of a gain of the transfer function and one by an adjustment parameter that is modulated as a function of the scene classification information;
applying the transfer function to pixel values of the image to produce a tone mapped image; and
outputting the tone mapped image.

15. The method of claim 14 , wherein the scene classification information is determined by an automatic white balance module configured to apply white balance correction to the image.

16. The method of claim 14 , comprising:
determining a blue sky indication that reflects a proportion of a scene depicted in the image that includes blue sky; and
modifying the transfer function based on the blue sky indication to reduce a gain of the transfer function.

17. The method of claim 16 , wherein the blue sky indication is determined by an automatic white balance module configured to apply white balance correction to the image.

18. The method of claim 14 , comprising:
determining a faces indication that reflects a proportion of a scene depicted in the image that includes one or more human faces; and
modifying the transfer function based on the faces indication to reduce a gain of the transfer function.

19. The method of claim 14 , comprising:
transposing the transfer function from a post-gamma curve space to a linear space.

20. The method of claim 14 , comprising:
adjusting the transfer function to remove above-threshold amplification of noise resulting from an application of the transfer function to the image.

21. The method of claim 14 , comprising:
computing a derivative of the transfer function;
adjusting one of a high threshold area and a low threshold area of the derivative of the transfer function such that the high threshold area and the low threshold area are equal;
modifying the derivative

Assignees

• Gopro Inc

Inventors

• Douady-Pleven Bruno César
• Kraak Michael Serge Andre
• Guérin Guillaume Matthieu
• Veit Thomas Nicolas Emmanuel

Classifications

• H04N23/88

  for colour balance, e.g. white-balance circuits or colour temperature control · CPC title

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N23/60

  Control of cameras or camera modules · CPC title

• H04N25/61

  the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4" · CPC title

• H04N1/603

  controlled by characteristics of the picture signal generator or the picture reproducer · CPC title