Methods and systems for inverse tone mapping

Patent metadata

Field	Value
Publication number	US-2016358319-A1
Application number	US-201415035477-A
Country	US
Kind code	A1
Filing date	Nov 19, 2014
Priority date	Nov 22, 2013
Publication date	Dec 8, 2016
Grant date	—

Abstract

Novel methods and systems for inverse tone mapping are disclosed. A scene can be analyzed to obtain highlight detection from bright light sources and specular reflections. An inverse tone mapping curve can be calculated based on the lower dynamic range and higher dynamic range displays. Multi-scale filtering can be applied to reduce noise or artifacts.

First claim

What is claimed is:

1 . A method to perform inverse tone mapping, the method comprising:
providing, by a computer, a luminance of an image; detecting, by a computer, highlights in the luminance, thereby obtaining a highlight detection parameter; estimating, by a computer, a mid-value of the luminance, thereby obtaining a mid-value luminance parameter; providing, by a computer, minimum and maximum luminance values for a lower dynamic range display and for a higher dynamic range display; calculating, by a computer, an inverse tone mapping curve, based on the highlight detection parameter, the mid-value luminance parameter, and the minimum and maximum luminance values for the lower dynamic range display and the higher dynamic range display; and applying, by a computer, the inverse tone mapping curve to the luminance as a global mapping operator, thereby obtaining an inversely mapped luminance.

2 . The method of claim 1 , wherein the detecting highlights comprises detecting highlights from brighter light sources and from specular reflections.

3 . The method of claim 1 , wherein the detecting highlights comprises estimating a probability of highlighted pixels in the image.

4 . The method of claim 1 , wherein the highlight detection parameter and the mid-value luminance parameter are scene-adaptive.

5 . The method of claim 1 , wherein the image is a frame in a video.

6 . The method of claim 5 , further comprising recursive filtering between adjacent frames in the video.

7 . The method of claim 1 , wherein the lower dynamic range display or the higher dynamic range display is a virtual display, and wherein the virtual display has a different maximum luminance value.

8 . The method of claim 7 , further comprising:
adding, by a computer, virtual highlights to the luminance, wherein the virtual highlights have a higher luminance level; displaying, by a computer, the image with virtual highlights on the virtual display; and removing, by a computer, the virtual highlights.

9 . The method of claim 1 , wherein the detecting highlights is based on a dark channel information and two threshold values.

10 . The method of claim 1 , wherein the detecting highlights is based on heuristic criteria, perception-based criteria, or a multi-scale approach.

11 . The method of claim 1 , wherein the calculating comprises:
obtaining, by a computer, a mid-value luminance of a lower dynamic range image; and estimating, by a computer, a mid-value luminance of a higher dynamic range image, based on the mid-value luminance of the lower dynamic range image.

12 . The method of claim 1 , further comprising applying a multi-scale filter to the inversely mapped luminance, thereby obtaining a filtered inversely mapped luminance.

13 . The method of claim 12 , wherein the multi-scale filter comprises a Gaussian filter, a bilateral filter, or a guided filter.

14 . The method of claim 12 , further comprising blending, by a computer, the inversely mapped luminance with the filtered inversely mapped luminance, thereby obtaining a blended inversely mapped luminance.

15 . The method of claim 14 , wherein the blending comprises:
defining, by a computer, a first threshold parameter and a second threshold parameter, wherein the second threshold parameter is greater than the first threshold parameter; iterating, by a computer, between pixels of the image, wherein the iterating is for at least one pixel and wherein the iterating executes one of the following steps
i. if a slope of the inverse tone mapping curve at a luminance value of the at least one pixel is less than the first threshold parameter, the blending for the at least one pixel considers only the inversely tone mapped luminance for the at least one pixel;
ii. if a slope of the inverse tone mapping curve at a luminance value of the at least one pixel is more than the second threshold parameter, the blending for the at least one pixel considers only the filtered inversely mapped luminance for the at least one pixel; or
iii. if a slope of the inverse tone mapping curve at a luminance value of the at least one pixel is more than the first threshold parameter and less than the second threshold parameter, the blending for the at least one pixel is linearly interpolated between the inversely tone mapped luminance and the filtered inversely mapped luminance for the at least one pixel, the slope

16 . The method of claim 15 , further comprising mapping the higher frequency luminance pixels separately.

17 . A system comprising:
a decoder, configured to receive an encoded lower dynamic range image or video and output a decoded lower dynamic range image or video; an inverse tone mapping module, configured to receive the decoded lower dynamic range image or video and output an inversely tone mapped image or video, wherein the inverse tone mapping is carried out according to the method of claim 1 .

18 . A system comprising:
at least one lower dynamic range camera, configured to output a lower dynamic range image or video; at least one higher dynamic range camera, configured to output a higher dynamic range image or video; an inverse tone mapping module, configured to receive the lower dynamic range image or video and output an inversely tone mapped image or video, wherein the inverse tone mapping is carried out according to the method of claim 1 ; a higher dynamic range display, configured to receive the higher dynamic range image or video and the inversely tone mapped image or video.

Assignees

• Dolby Laboratories Licensing Corp

Inventors

• Xu Ning
• Chen Tao
• Crenshaw James E
• Kunkel Timo
• Lee Bongsun

Classifications

• G06T5/40

  using histogram techniques · CPC title

• G06T7/408

  Physics · mapped topic

• G06T2207/20208

  High dynamic range [HDR] image processing · CPC title

• H04N1/6027

  Correction or control of colour gradation or colour contrast (H04N1/6058 takes precedence) · CPC title

• G06T5/007 Primary

  Physics · mapped topic