Generation of three-dimensional (3D) lookup table for tone mapping or other image processing functions

Patent metadata

Field	Value
Publication number	US-12430731-B2
Application number	US-202218046117-A
Country	US
Kind code	B2
Filing date	Oct 12, 2022
Priority date	Feb 9, 2022
Publication date	Sep 30, 2025
Grant date	Sep 30, 2025

Abstract

A method includes obtaining an image and a gain map associated with the image. The method also includes identifying image patches in the image and corresponding gain map patches in the gain map. Different image patches are centered around different anchor points in the image. The method further includes, for each image patch and its corresponding gain map patch, generating an intensity-gain curve for the associated anchor point. The intensity-gain curve specifies (i) gain values based on the corresponding gain map patch for intensity values up to a threshold intensity value and (ii) gain values based on one or more input parameters for intensity values above the threshold intensity value. In addition, the method includes combining the intensity-gain curves to generate a 3D lookup table, which identifies the gain values for the anchor points in the image at each of multiple intensity values.

First claim

What is claimed is:

1. A method comprising:
obtaining an image and a gain map associated with the image;
identifying image patches in the image and corresponding gain map patches in the gain map, wherein different image patches are centered around different anchor points in the image;
for each image patch and its corresponding gain map patch, generating an intensity-gain curve for the associated anchor point, wherein the intensity-gain curve specifies (i) gain values based on the corresponding gain map patch for intensity values up to a threshold intensity value and (ii) gain values based on one or more input parameters for intensity values above the threshold intensity value;
for each of at least one of the intensity-gain curves, replacing a portion of the intensity-gain curve with a parameterized tail, the portion having intensity values above the threshold intensity value; and
combining the intensity-gain curves to generate a three-dimensional (3D) lookup table, the 3D lookup table identifying the gain values for the anchor points in the image at each of multiple intensity values.

2. The method of claim 1 , wherein, for each intensity-gain curve, the one or more input parameters define knee points for a portion of the intensity-gain curve above the threshold intensity value, the knee points used to define the parameterized tail and smooth decay of the intensity-gain curve.

3. The method of claim 1 , wherein, for each intensity-gain curve, the gain value associated with each intensity value up to the threshold intensity value comprises a mean, a maximum, a minimum, or a median of the gain values in the corresponding gain map patch.

4. The method of claim 1 , wherein combining the intensity-gain curves to generate the 3D lookup table comprises:
performing low-pass filtering of the intensity-gain curves across an intensity dimension to provide smooth tone transitions between adjacent intensity values.

5. The method of claim 1 , wherein combining the intensity-gain curves to generate the 3D lookup table comprises:
performing low-pass filtering of the intensity-gain curves across a spatial dimension to provide smooth spatial tone transitions.

6. The method of claim 1 , further comprising:
storing the 3D lookup table in association with raw image data representing the image, wherein the 3D lookup table automatically adjusts a tone-mapping of the raw image data when the raw image data is loaded into a viewer.

7. The method of claim 6 , wherein the tone-mapping of the raw image data is adjusted by:
identifying four coordinates in the 3D lookup table, the four coordinates associated with four anchor points around an arbitrary coordinate of the raw image data;
identifying gain values contained in the 3D lookup table at the four coordinates;
performing interpolation of the identified gain values to generate an interpolated gain value; and
applying the interpolated gain value to at least one pixel of the raw image data at the arbitrary coordinate.

8. An apparatus comprising:
at least one processor configured to:
obtain an image and a gain map associated with the image;
identify image patches in the image and corresponding gain map patches in the gain map, wherein different image patches are centered around different anchor points in the image;
for each image patch and its corresponding gain map patch, generate an intensity-gain curve for the associated anchor point, wherein the intensity-gain curve specifies (i) gain values based on the corresponding gain map patch for intensity values up to a threshold intensity value and (ii) gain values based on one or more input parameters for intensity values above the threshold intensity value;
for each of at least one of the intensity-gain curves, replace a portion of the intensity-gain curve with a parameterized tail, the portion having intensity values above the threshold intensity value; and
combine the intensity-gain curves to generate a three-dimensional (3D) lookup table, the 3D lookup table identifying the gain values for the anchor points in the image at each of multiple intensity values.

9. The apparatus of claim 8 , wherein, for each intensity-gain curve, the one or more input parameters define knee points for a portion of the intensity-gain curve above the threshold intensity value, the at least one processor configured to use the knee points to define the parameterized tail and smooth decay of the intensity-gain curve.

10. The apparatus of claim 8 , wherein, for each intensity-gain curve, the gain value associated with each intensity value up to the threshold intensity value comprises a mean, a maximum, a minimum, or a median of the gain values in the corresponding gain map patch.

11. The apparatus of claim 8 , wherein, to combine the intensity-gain curves to generate the 3D lookup table, the at least one processor is configured to perform low-pass filtering of the intensity-gain curves across an intensity dimension to provide smooth tone transitions between adjacent intensity values.

12. The apparatus of claim 8 , wherein, to combine the intensity-gain curves to generate the 3D lookup table, the at least one processor is configured to perform low-pass filtering of the intensity-gain curves across a spatial dimension to provide smooth spatial tone transitions.

13. The apparatus of claim 8 , wherein the at least one processor is further configured to store the 3D lookup table in association with raw image data representing the image.

14. The apparatus of claim 13 , wherein:
the at least one processor is further configured to adjust a tone-mapping of the raw image data when the raw image data is loaded into a viewer; and
to adjust the tone-mapping of the raw image data, the at least one processor is configured to:
identify four coordinates in the 3D lookup table, the four coordinates associated with four anchor points around an arbitrary coordinate of the raw image data;
identify gain values contained in the 3D lookup table at the four coordinates;
perform interpolation of the identified gain values to generate an interpolated gain value; and
apply the interpolated gain value to at least one pixel of the raw image data at the arbitrary coordinate.

15. A non-transitory computer readable medium containing instructions that when executed cause at least one processor to:
obtain an image and a gain map associated with the image;
identify image patches in the image and corresponding gain map patches in the gain map, wherein different image patches are centered around different anchor points in the image;
for each image patch and its corresponding gain map patch, generate an intensity-gain curve for the associated anchor point, wherein the intensity-gain curve specifies (i) gain values based on the corresponding gain map patch for intensity values up to a threshold intensity value and (ii) gain values based on one or more input parameters for intensity values above the threshold intensity value;
for each of at least one of the intensity-gain curves, replace a portion of the intensity-gain curve with a parameterized tail, the portion having intensity values above the threshold intensity value; and
combine the intensity-gain curves to generate a three-dimensional (3D) lookup table, the 3D lookup table identifying the gain values for the anchor points in the image at each of multiple intensity values.

16. The non-transitory computer readable medium of claim 15 , wherein, for each intensity-gain curve, the one or more input parameters define knee points for a portion of the intensity-gain curve above the threshold intensity value,

Assignees

• Samsung Electronics Co Ltd

Inventors

• Godaliyadda Gunawath Dilshan
• Glotzbach John W
• Sheikh Hamid R

Classifications

• G06T5/70

  Denoising; Smoothing · CPC title

• G06T5/20

  using local operators · CPC title

• G06T2207/20208

  High dynamic range [HDR] image processing · CPC title

• H04N9/69

  for modifying the colour signals by gamma correction (controlling camera response for colour signals H04N23/83) · CPC title

• G06T2207/10024

  Color image · CPC title