Image sharpening

Patent metadata

Field	Value
Publication number	US-11727540-B2
Application number	US-202017139880-A
Country	US
Kind code	B2
Filing date	Dec 31, 2020
Priority date	Dec 31, 2020
Publication date	Aug 15, 2023
Grant date	Aug 15, 2023

Abstract

Various image sharpening techniques are disclosed. For example, a method for image sharpening includes obtaining, using at least one sensor of an electronic device, an image that includes visual content. The method also includes generating an edge map that indicates edges of the visual content within the image. The method further includes applying a high-pass signal and an adaptive gain based on the edge map to sharpen the image. The method also includes generating a bright halo mask and a dark halo mask based on the edge map, where the bright halo mask indicates an upper sharpening limit and the dark halo mask indicates a lower sharpening limit. In addition, the method includes modifying a level of sharpening at one or more of the edges within the sharpened image to provide halo artifact reduction based on the bright halo mask and the dark halo mask.

First claim

What is claimed is:

1. A method for image sharpening, the method comprising:
obtaining, using at least one sensor of an electronic device, an image that includes visual content;
generating an edge map that indicates edges of the visual content within the image;
applying a high-pass signal and an adaptive gain based on the edge map to sharpen the image;
generating a bright halo mask and a dark halo mask based on the edge map, wherein the bright halo mask indicates an upper sharpening limit and the dark halo mask indicates a lower sharpening limit; and
modifying a level of sharpening at one or more of the edges within the sharpened image to provide halo artifact reduction based on the bright halo mask and the dark halo mask.

2. The method of claim 1 , further comprising:
after the high-pass signal is applied to the image, removing noise within the image to smooth the image;
wherein applying the adaptive gain comprises applying the adaptive gain to the smoothed image; and
wherein modifying the level of sharpening comprises reducing an amount of sharpening at the edges within the sharpened image based on the bright halo mask and the dark halo mask.

3. The method of claim 1 , wherein applying the adaptive gain comprises:
applying a specified level of gain at a center of an edge as indicated by the edge map; and
reducing the specified level of gain incrementally as distance from the center of the edge increases.

4. The method of claim 1 , wherein generating the bright halo mask and the dark halo mask comprises:
selecting a specified edge from the edge map;
identifying a location of the specified edge in the image; and
determining that a first side of the specified edge is brighter than a second side of the specified edge; and
wherein the bright halo mask corresponds to the first side of the specified edge and the dark halo mask corresponds to the second side of the specified edge.

5. The method of claim 1 , further comprising:
after the high-pass signal is applied to the image, removing noise within the image;
wherein generating the bright halo mask and the dark halo mask comprises:
generating the bright halo mask to indicate locations where the high-pass signal is positive; and
generating the dark halo mask to indicate locations where the high-pass signal is negative; and

wherein modifying the level of sharpening comprises reducing the level of sharpening at one or more edges according to the bright halo mask and the dark halo mask.

6. The method of claim 1 , further comprising:
identifying an edge density value at a portion of the image;
comparing the edge density value to a threshold; and
reducing halo control at the portion of the image based on a comparison of the edge density value to the threshold.

7. The method of claim 6 , further comprising:
performing text segmentation using the image;
determining that the portion of the image includes text based on the text segmentation; and
maintaining the halo control around a region of the text.

8. A method for image sharpening, the method comprising:
obtaining an image of a scene using at least one sensor of an electronic device;
applying multiple Gaussian filtering operations to the obtained image, the Gaussian filtering operations changing different frequencies of an input signal of the obtained image;
generating multiple intermediate sharpened images using the obtained image, wherein generating the multiple intermediate sharpened images comprises, for each of the multiple Gaussian filtering operations, performing non-local filtering for noise reduction to generate one of the intermediate sharpened images; and
combining portions of the multiple intermediate sharpened images and the obtained image to generate a sharpened image of the scene.

9. The method of claim 8 , wherein applying the multiple Gaussian filtering operations comprises:
performing a first Gaussian filtering operation based on the input signal of the obtained image to generate a first filtered signal; and
performing a second Gaussian filtering operation based on the first filtered signal.

10. A method for image processing, the method comprising:
obtaining multiple image frames of a scene using at least one under-display camera (UDC) of an electronic device, the multiple image frames including a first image frame and a second image frame that are captured using different exposures;
combining the first image frame and the second image frame to generate a blended image of the scene and a map, wherein the map is based on at least one of the multiple image frames;
identifying a correction signal based on a comparison of the blended image and an output of a point spread function inversion (PSFI) process for increasing sharpness and removing artifacts;
modifying the correction signal based on the map; and
generating an image of the scene based on the modified correction signal and the blended image of the scene.

11. The method of claim 10 , wherein the correction signal represents differences between an input to the PSFI process and the output of the PSFI process.

12. The method of claim 10 , further comprising:
modifying the map by normalizing the map; and
modifying the correction signal by performing element-wise multiplication with the modified map.

13. The method of claim 10 , wherein:
the map comprises a motion map;
combining the first image frame and the second image frame comprises aligning the first image frame and the second image frame; and
generating the motion map occurs as part of the aligning of the first image frame and the second image frame, wherein the motion map indicates motion between the first image frame and the second image frame.

14. The method of claim 10 , wherein:
the map comprises a gain map;
generating the gain map comprises:
aligning the first image frame with the second image frame; and
processing the aligned image frames using tone mapping to generate the gain map; and

modifying the correction signal is based on an inverse of the gain map.

15. The method of claim 10 , wherein the map comprises at least one of a gain map and a motion map.

16. An electronic device comprising:
a display;
at least one image sensor positioned under the display; and
at least one processor operatively connected to the display and the at least one image sensor, the at least one processor configured to:
obtain multiple image frames of a scene using the at least one image sensor, the multiple image frames including a first image frame and a second image frame that are captured using different exposures;
combine the first image frame and the second image frame to generate a blended image of the scene and a map, wherein the map is based on at least one of the multiple image frames;
identify a correction signal based on a comparison of the blended image and an output of a point spread function inversion (PSFI) process for increasing sharpness and removing artifacts;
modify the correction signal based on the map; and
generate an image of the scene based on the modified correction signal and the blended image of the scene.

17. The electronic device of claim 16 , wherein the correction signal represents differences between an input to the PSFI process and the output of the PSFI process.

18. The electronic device of claim 16 , wherein the at least one processor is further configured to:
normalize the map; and
perform element-wise multiplication with the normalized map to modify the correction signal.

19. The electronic device of claim 16

Assignees

• Samsung Electronics Co Ltd

Inventors

• Tong Zhen
• Xu Yibo
• Glotzbach John W
• Sheikh Hamid R
• Zhen Ruiwen
• Choi Baekdu

Classifications

• G06T2207/20012

  Locally adaptive · CPC title

• G06T5/73 Primary

  Deblurring; Sharpening · CPC title

• G06T5/90

  Dynamic range modification of images or parts thereof · CPC title

• G06T5/70

  Denoising; Smoothing · CPC title

• G06T5/92

  based on global image properties · CPC title