Image processing method and apparatus

Patent metadata

Field	Value
Publication number	US-11301970-B2
Application number	US-201916549176-A
Country	US
Kind code	B2
Filing date	Aug 23, 2019
Priority date	May 2, 2019
Publication date	Apr 12, 2022
Grant date	Apr 12, 2022

Abstract

An image processing method acquires an image, restores a saturated region in which a pixel in the image has a first reference value based on a first illuminance component of the image, enhances a dark region in which a value of a pixel in the image is less than a second reference value based on the restored saturated region and the first illuminance component, and outputs a dark region-enhanced image.

First claim

What is claimed is:

1. An image processing method, comprising:
acquiring an image;
obtaining a first illuminance component of the image from an offset-compensated image;
restoring a saturated region in which a first pixel in the image has a first reference value based on the first illuminance component of the image by using a pixel value of a subpixel having a lowest sensitivity among red (R), green (G), and blue (B) subpixels of each pixel included in the saturated region;
calculating an increment of each pixel value of a dark region-enriched image relative to pixel values of the image by using an over curve, wherein the increment of each pixel value is adjusted using the over curve;
enhancing a dark region in which a value of a second pixel in the image is less than a second reference value based on the restored saturated region and the first illuminance component, by calculating a local contrast factor based on an increment of a corresponding pixel and an increment of a surrounding pixel, and adjusting the increment of the corresponding pixel based on the local contrast factor; and
outputting a dark region-enhanced image.

2. The image processing method of claim 1 , wherein the restoring of the saturated region based on the subpixel having the lowest sensitivity and the first illuminance component of the image comprises:
obtaining a luminance component of the image;
restoring a luminance component of the saturated region by assuming a linear distribution of the image based on a first illuminance component of the subpixel having the lowest sensitivity and the luminance component;
obtaining residual information from an image of the subpixel having the lowest sensitivity and the restored luminance component using a guided filter; and
restoring the saturated region by applying the residual information to the luminance component.

3. The image processing method of claim 2 , wherein the restoring of the luminance component of the saturated region comprises restoring the luminance component of the saturated region based on the subpixel having the lowest sensitivity and the first illuminance component of the image, in response to the subpixel having the lowest sensitivity being unsaturated.

4. The image processing method of claim 1 , wherein the enhancing comprises:
enhancing the dark region based on the over curve; and
performing a dynamic range compression (DRC) with respect to a dark region-enhanced image.

5. The image processing method of claim 4 , wherein the enhancing of the dark region based on the over curve comprises:
enriching the dark region using the over curve to generate the dark region-enriched image.

6. The image processing method of claim 1 , further comprising:
adjusting a contrast of the dark region-enhanced image,
wherein the outputting of the dark region-enhanced image comprises outputting a contrast-adjusted image.

7. The image processing method of claim 6 , wherein the adjusting of the contrast comprises:
adjusting a local contrast of the dark region-enhanced image using a histogram equalization (HE) based local curve; and
adjusting a global contrast of the dark region-enhanced image by readjusting the local curve based on a second illuminance component extracted from the dark region-enhanced image.

8. The image processing method of claim 7 , wherein the adjusting of the local contrast comprises:
segmenting the dark region-enhanced image into blocks;
estimating the HE based local curve for each of the blocks; and
adjusting a contrast for each of the blocks by adjusting the local curve based on a sum of weights corresponding to distances from pixels for each of the blocks to the local curve.

9. The image processing method of claim 7 , wherein the adjusting of the global contrast comprises:
extracting the second illuminance component from the dark region-enhanced image; and
adjusting the global contrast by readjusting the local curve based on the second illuminance component.

10. The image processing method of claim 1 , further comprising:
compensating for an offset corresponding to a pixel value less than the second reference value in the image based on a cumulative distribution function (CDF).

11. The image processing method of claim 10 , wherein the compensating comprises:
setting pixel values of a dark region in which a pixel is estimated to have a value less than the second reference value among values of the pixels included in the image, as the offset and removing the offset; and
estimating the CDF by linearly stretching pixels remaining after the offset is removed from the image.

12. The image processing method of claim 10 , wherein the obtaining comprises estimating the first illuminance component of the image by passing the offset-compensated image separately through a cross bilateral filter (CBF) based first local filter and a just noticeable difference (JND) based second local filter.

13. The image processing method of claim 12 , wherein the obtaining comprises:
generating a global blur image corresponding to the image by passing the offset-compensated image through the CBF based first local filter;
generating a local blur image corresponding to an edge region of the image by passing the offset-compensated image through the JND based second local filter; and
obtaining the first illuminance component by blending the global blur image and the local blur image based on a weight.

14. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the image processing method of claim 1 .

15. An image processing apparatus, comprising: one or more processors configured to:
obtain a first illuminance component of an image from an offset-compensated image;
restore a saturated region n which a first pixel in the image has a first reference value based on the first illuminance component of the image by using a pixel value of a sub pixel having a lowest sensitivity among red (R), green (G), and blue (B) subpixels of each pixel included in the saturated region;
calculate an increment of each pixel value of a dark region-enriched image relative to pixel values of the image by using an over curve, wherein the increment of each pixel value is adjusted using the over curve;
enhance a dark region in which a value of a second pixel in the image is less than a second reference value based on the restored saturated region and the first illuminance component, by calculating a local contrast factor based on an increment of a corresponding pixel and an increment of a surrounding pixel, and adjusting the increment of the corresponding pixel based on the local contrast factor;
and output a dark region-enhanced image.

16. The image processing apparatus of claim 15 , wherein the one or more processors are further configured to:
obtain a luminance component of the image,
restore a luminance component of the saturated region by assuming a linear distribution of the image based on a first illuminance component of the subpixel having the lowest sensitivity and the luminance component,
obtain residual information from an image of the subpixel having the lowest sensitivity and the restored luminance component using a guided filter, and
restore the saturated region by applying the residual information to the luminance component.

17. The image processing apparatus of claim 15 , wherein the one or more processors are further configured to:
enrich the dark region using the over curve to generate the dark region-enriched image; and
perform a dynamic range com

Assignees

• Samsung Electronics Co Ltd
• Univ Yonsei Iacf

Inventors

• Jung Hwiryong
• Kang Moon Gi
• Jee Seung Hoon
• Kim Min Sub
• Kim Wooshik
• Moon Hyewon
• Lee Keechang
• Hong Sunghoon

Classifications

• G06T5/94 Primary

  based on local image properties, e.g. for local contrast enhancement · CPC title

• G06T5/90 Primary

  Dynamic range modification of images or parts thereof · CPC title

• G06T5/40

  using histogram techniques · CPC title

• H04N1/60

  Colour correction or control {(H04N1/54 takes precedence)} · CPC title

• G06T2207/20208

  High dynamic range [HDR] image processing · CPC title