Global approximation to spatially varying tone mapping operators

Patent metadata

Field	Value
Publication number	US-9626744-B2
Application number	US-201514840843-A
Country	US
Kind code	B2
Filing date	Aug 31, 2015
Priority date	Nov 21, 2012
Publication date	Apr 18, 2017
Grant date	Apr 18, 2017

Abstract

Techniques to generate global tone-mapping operators (G-TMOs) that, when applied to high dynamic range images, visually approximate the use of spatially varying tone-mapping operators (SV-TMOs) are described. The disclosed G-TMOs provide substantially the same visual benefits as SV-TMOs but do not suffer from spatial artifacts such as halos and are, in addition, computationally efficient compared to SV-TMOs. In general, G-TMOs may be identified based on application of a SV-TMO to a down-sampled version of a full-resolution input image (e.g., a thumbnail). An optimized mapping between the SV-TMO's input and output constitutes the G-TMO. It has been unexpectedly discovered that when optimized (e.g., to minimize the error between the SV-TMO's input and output), G-TMOs so generated provide an excellent visual approximation to the SV-TMO (as applied to the full-resolution image).

First claim

The invention claimed is:

1. A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to:
obtain a grayscale version of a high dynamic range (HDR) color input image;
downsample the grayscale version of the HDR color input image to generate a down-sampled grayscale input image;
apply a spatially varying tone mapping operator (SV-TMO) to the down-sampled grayscale input image to generate a down-sampled grayscale output image;
determine a global tone mapping operator (G-TMO) according to, at least in part, a Pool-Adjacent-Violators-Algorithm (PAVA); and
apply the G-TMO to the grayscale version of the HDR color input image to generate a low dynamic range (LDR) grayscale output image.

2. The non-transitory program storage device of claim 1 , wherein the instructions to determine the G-TMO further comprises instructions to cause the one or more processors to:
determine a first G-TMO according to the PAVA; and
apply a smoothing filter to the first G-TMO to generate the G-TMO.

3. The non-transitory program storage device of claim 1 , wherein the instructions to determine the G-TMO comprise instructions to cause the one or more processors to determine a mapping between each pixel in the down-sampled grayscale input image to a corresponding pixel in the down-sampled grayscale output image, wherein the mapping is selected to minimize a specified error criterion.

4. The non-transitory program storage device of claim 3 , wherein the specified error criterion comprises a root means squares error criteria.

5. The non-transitory program storage device of claim 1 , further comprising instructions to cause the one or more processors to convert the LDR grayscale output image to a LDR color output image.

6. The non-transitory program storage device of claim 2 , wherein each pixel in the color output image is based, in part, on a ratio of the corresponding pixels in the grayscale output image and the grayscale version of the color input image.

7. The non-transitory program storage device of claim 1 , wherein the instructions to obtain the grayscale version of the HDR color input image comprise instructions to cause the one or more processors to obtain a brightness channel of the HDR color input image, wherein the HDR color input image comprises a brightness channel and one or more chrominance channels.

8. The non-transitory program storage device of claim 1 , wherein the instructions to generate the down-sampled grayscale input image comprise instructions to cause the one or more processors to generate a thumbnail of the grayscale version of the HDR color input image.

9. The non-transitory program storage device of claim 1 , wherein the instructions to determine the first G-TMO according to the PAVA comprise instructions to cause the one or more processors to quantize values from the down-sampled grayscale input image into a specified number of levels, wherein the specified number of levels is less than the number of possible brightness levels that a pixel in the down-sampled grayscale input image may have.

10. The non-transitory program storage device of claim 1 , wherein the instructions to generate the down-sampled grayscale output image comprise instructions to cause the one or more processors to:
apply the G-TMO to the grayscale version of the HDR color input image to generate a first grayscale image; and
perform detail recovery operations on the first grayscale image to generate a grayscale output image.

11. The non-transitory program storage device of claim 10 , wherein the instructions to perform the detail recovery operations comprise instructions to cause the one or more processors to apply a bilateral filter to a combination of the grayscale version of the HDR color input image and the first grayscale image.

12. The non-transitory program storage device of claim 10 , wherein the instructions to generate the grayscale output image comprise instructions to cause the one or more processors to:
perform detail recovery operations on the first grayscale image to generate a second grayscale image; and
apply an unsharp mask to the second grayscale image to generate a grayscale output image.

13. An electronic device, comprising:
a display element;
a memory operatively coupled to the display element; and
one or more processing units operatively coupled to the display element and the memory, and adapted to execute instructions stored in the memory to:
obtain a grayscale version of a high dynamic range (HDR) color input image;
downsample the grayscale version of the HDR color input image to generate a down-sampled grayscale input image;
apply a spatially varying tone mapping operator (SV-TMO) to the down-sampled grayscale input image to generate a down-sampled grayscale output image;
determine a global tone mapping operator (G-TMO) according to, at least in part, a Pool-Adjacent-Violators-Algorithm (PAVA); and
apply the G-TMO to the grayscale version of the HDR color input image to generate a low dynamic range (LDR) grayscale output image.

14. The electronic device of claim 13 , wherein the one or more processing units are adapted to execute instructions to:
determine a first G-TMO according to the PAVA; and
apply a smoothing filter to the first G-TMO to generate the G-TMO.

15. The electronic device of claim 13 , wherein the one or more processing units are adapted to execute instructions to determine a mapping between each pixel in the down-sampled grayscale input image to a corresponding pixel in the down-sampled grayscale output image, wherein the mapping is selected to minimize a specified error criterion.

16. The electronic device of claim 15 , wherein the specified error criterion comprises a root means squares error criteria.

17. The electronic device of claim 13 , wherein the one or more processing units are adapted to execute instructions to convert the LDR grayscale output image to a LDR color output image.

18. The electronic device of claim 17 , wherein each pixel in the color output image is based, in part, on a ratio of the corresponding pixels in the grayscale output image and the grayscale version of the color input image.

19. The electronic device of claim 13 , wherein the one or more processing units are adapted to execute instructions to obtain a brightness channel of the HDR color input image, wherein the HDR color input image comprises a brightness channel and one or more chrominance channels.

20. The electronic device of claim 13 , wherein the one or more processing units are adapted to execute instructions to generate a thumbnail of the grayscale version of the HDR color input image.

21. The electronic device of claim 13 , wherein the one or more processing units are adapted to execute instructions to quantize values from the down-sampled grayscale input image into a specified number of levels, wherein the specified number of levels is less than the number of possible brightness levels that a pixel in the down-sampled grayscale input image may have.

22. The electronic device of claim 13 , wherein the one or more processing units are adapted to execute instructions to:
apply the G-TMO to the grayscale version of the HDR color input image to generate a first grayscale image; and
perform detail recovery operations on the first grayscale image to generate a grayscale output image.

23. The electronic device of claim 22 , wherein the one or more processing units are adapted to execu

Assignees

• Apple Inc

Inventors

• Finlayson Graham D
• Singnoo Jakkarin

Classifications

• G06T5/40

  using histogram techniques · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T2207/20208

  High dynamic range [HDR] image processing · CPC title

• G06T2207/20192

  Edge enhancement; Edge preservation · CPC title

• G06T5/002 Primary

  Physics · mapped topic