Enhanced high-dynamic-range imaging and tone mapping

Patent metadata

Field	Value
Publication number	US-2021035273-A1
Application number	US-201916526902-A
Country	US
Kind code	A1
Filing date	Jul 30, 2019
Priority date	Jul 30, 2019
Publication date	Feb 4, 2021
Grant date	—

Abstract

The various embodiments of the present disclosure are directed towards methods for tone mapping High-Dynamic-Range (HDR) image data, as well as controlling the brightness of the image encoded by HDR the image data and/or the tone-mapped image data. HDR image is captured. A tone mapping function for the HDR image data is generated. To generate the tone mapping function, control points are dynamically determined based on an analysis of the HDR image data. The tone mapping function is fit to the control points. The tone mapping function is a non-linear function, and is described by a curve in a plane. The shape of the curve is constrained by a line generated from a portion of the control points. The tone mapping function is applied to the HDR image data. A color-compression is applied to the tone mapped image data to generate Standard Dynamic Range or Low Dynamic Range image data.

First claim

What is claimed is:

1 . A method comprising:
receiving high dynamic range (HDR) source image data representative of a source image; determining tone control points based on source pixel values of the source image data; determining a gain value based on a first tone point and at least one other tone control point of the tone control points; determining a tone mapping function based on the tone control points and the gain value; and generating target image data representative of at least one target image, wherein the target image data includes target pixel values defined by the tone mapping function.

2 . The method of claim 1 , further comprising, subsequent to receiving the source image data, receiving additional HDR source image data representative of an additional source image, wherein the generating the target image data includes applying the tone mapping function to pixel values of the additional source image data.

3 . The method of claim 1 , wherein the tone mapping function:
maps a first scalar value of a second tone point to a second scalar value; maps a third scalar value of the first tone point to a fourth scalar value; and maps a fifth scalar value of the third tone point to a sixth scalar value, wherein the first tone point has a corresponding pixel value that is substantially equidistant between respective pixel values corresponding to the second and third tone points.

4 . The method of claim 1 , wherein a rate of change of the tone mapping function, evaluated at a first scalar value of the first tone point, is substantially equivalent to the gain value.

5 . The method of claim 1 , further comprising:
determining a region-of-interest (ROI) of the source image; and determining the tone control points based on a portion of the source image data that corresponds the ROI of the source image.

6 . The method of claim 1 , wherein the determining the tone control points includes:
generating log-transformed image data values based on the source pixel values; determining an average value of the log-transformed image data values; and determining the first tone point based on an exponentiation of the average value of the log-transformed image data values.

7 . The method of claim 1 , wherein the determining tone control points includes:
generating filtered source image data by applying each of a first tone filter and a second tone filter to the source image data; and determining the first tone point based on the filtered source image data.

8 . The method of claim 1 , wherein the determining tone control points includes:
determining a first subset of the source pixel values, wherein a value for each pixel included in the first subset is greater than the value for each pixel of the source pixels values excluded from the first subset; determining a second subset of the source pixel values, wherein the value for each pixel included in the second subset is less than the value for each pixel of the source pixel values excluded from the second subset; determining a third tone point based on the first subset of the source pixel values, the third tone point having a higher corresponding pixel value than the first tone point; and determining a second tone point based on the second subset of the source pixel values, the second tone point having a lower corresponding pixel value than the first tone point.

9 . The method of claim 1 , wherein the determining tone control points includes:
determining a second tone point based on a minimum value for the target pixel values, the second tone point having a lower corresponding pixel value than the first tone point; determining the first tone point based on a mid-tone value for the target pixel values; and determining the third tone point based on a maximum-value for the target pixel values, the third tone point having a greater corresponding pixel value than the first tone point.

10 . The method of claim 1 , wherein the source image data was captured by at least one camera device mounted on a vehicle.

11 . The method of claim 1 , further comprising:
employing a general processor of a camera device that captured the source image data to determine the tone mapping function; and employing a pipeline of an image processor (ISP) of the camera device to generate the target image data.

12 . The method of claim 1 , wherein the source image data was captured by at least one camera device mounted on a robot.

13 . The method of claim 1 , further comprising:
generating a plurality of statistical metrics of the source pixel values; and determining at least a portion of the tone control points based on the plurality of statistical metrics.

14 . The method of claim 1 :
generating color-compressed target data by applying a gamma-compression function to the target image data; and generating low dynamic range (SDR) image data based on the color-compressed target data.

15 . The method of claim 1 , further comprising:
clipping each pixel value of the source pixel values that is less than a flare-suppression threshold indicated by a second tone point, the second tone point having a lower corresponding pixel value than the first tone point; and clipping each pixel value in the source pixel values that is greater than a highlight-compression threshold indicated by a third tone point, the third tone point having a greater corresponding pixel value than the first tone point.

16 . A system comprising:
a processor device; and a computer-readable storage medium coupled with the processor device and having instructions stored thereon, which, when executed by the processor device, cause performance of actions including:
receiving first source image data representative of a first source image;
determining tone control points based on first source pixel values of the first source image;
determining a tone mapping function based on the tone control points;
receiving second source image data representative of a second source image; and
generating target image data by applying the tone mapping function to second source pixel values of the second source image.

17 . The method of claim 16 , wherein the system is incorporated in an autonomous vehicle, the autonomous vehicle comprising an image sensor that generates at least one of the first or second image data, wherein the at least one of the first or second image data is used as input to one or more deep neural networks producing outputs used to control the autonomous vehicle.

18 . The system of claim 16 , the actions further including:
employing a generalized-processor of a camera device to determine at least a portion of the tone control points; and employing an image-processor of the camera device to generate the target image data.

19 . A method comprising:
receiving high dynamic range (HDR) image data representative of an HDR image; determining tone control points based on pixel values of the HDR image; determining a gain value based on at least a portion of the tone control points; determining a tone mapping function based on the tone control points and the gain value; and generating lower dynamic range image data that is representative of a lower dynamic range image than the HDR image based on the tone mapping function and at least a portion of the pixel values of the HDR image.

20 . The method of claim 19 , wherein the determining the tone mapping function is such that a rate of change of the tone mapping function, evaluated at a first control point of

Assignees

• Nvidia Corp

Inventors

• Deng Yining
• Dujardin Eric
• Mirzaei Domabi Hamidreza
• Hwang Sung Hyun

Classifications

• H04N23/741 Primary

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N9/77

  Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase (circuits for matrixing H04N9/67) · CPC title

• G06T2207/20208

  High dynamic range [HDR] image processing · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T2207/10024

  Color image · CPC title