Apparatus and method for obtaining image employing color separation lens array

Patent metadata

Field	Value
Publication number	US-12457426-B2
Application number	US-202418414132-A
Country	US
Kind code	B2
Filing date	Jan 16, 2024
Priority date	Jan 17, 2023
Publication date	Oct 28, 2025
Grant date	Oct 28, 2025

Abstract

Provided is an apparatus for obtaining an image includes an image sensor and a signal processing unit that includes a demosaicing unit configured to reconstruct a green signal to have a full pixel resolution by using the input image, a sharpening filter unit configured to generate a first image by sharpening the reconstructed green signal for each preset direction, a direction image generation unit which generates a second image by removing a base band and extracting only a detail band, a gray detection unit configured to detect a gray region of the white balance-processed input image, an edge detection unit configured to detect an edge direction of the white balance-processed input image, and a selection unit configured to generate a third image by blending the first image and the second image, based on the detected gray region and the detected edge direction.

First claim

What is claimed is:

1. An apparatus for obtaining an image, comprising:
an image sensor comprising a color separation lens array configured to separate and concentrate incident light based on a color of the incident light; and
a signal processor configured to process an input image obtained by the image sensor,
wherein the signal processor configured to be implemented as:
a demosaicing unit configured to reconstruct a green signal to have a full pixel resolution based on the input image;
a sharpening filter unit configured to generate a first image by sharpening the reconstructed green signal for each preset direction;
a direction image generation unit configured to perform white balance processing on the input image, and generate a second image by removing a base band and extracting a detail band from the white balance-processed input image for each preset direction based on an unsharpening filtering algorithm;
a gray detection unit configured to perform white balance processing on the input image and detect a gray region of the white balance-processed input image;
an edge detection unit configured to perform white balance processing on the input image and detect an edge direction of the white balance-processed input image; and
a selection unit configured to generate a third image by blending the first image and the second image based on the detected gray region and the detected edge direction.

2. The apparatus for obtaining the image of claim 1 , wherein the image sensor comprises:
a sensor substrate comprising a plurality of light sensing cells configured to sense light, and
wherein the color separation lens array is on the sensor substrate and comprises a microstructure that forms a phase distribution configured to concentrate light of different wavelengths in light sensing cells, among the plurality of light sensing cells, adjacent to each other.

3. The apparatus for obtaining the image of claim 1 , wherein the image sensor comprises a Bayer pattern array.

4. The apparatus for obtaining an image of claim 1 , wherein the signal processor is further configured to be implemented as an adjustment unit configured to adjust a red signal and a blue signal, based on an amount of change of the green signal of the third image.

5. The apparatus for obtaining the image of claim 4 , wherein the signal processor is further configured to be implemented as a Bayer recombination unit configured to combine the green signal of the third image with the adjusted red signal and the adjusted blue signal to generate a fourth image having a Bayer pattern array.

6. The apparatus for obtaining the image of claim 1 , wherein the signal processor is further configured to be implemented as a phasor unit configured to detect an edge direction of the reconstructed green signal.

7. The apparatus for obtaining the image of claim 6 , wherein the selection unit is further configured to increase an edge resolution of the first image corresponding to the edge direction detected by the phasor unit based on an edge strength of the reconstructed green signal being greater than or equal to a preset value.

8. The apparatus for obtaining the image of claim 7 , wherein the selection unit is further configured to determine, by the gray detection unit, whether the gray region exists in the white balance-processed input image based on the edge strength of the reconstructed green signal being greater than or equal to the preset value.

9. The apparatus for obtaining the image of claim 8 , wherein the selection unit is further configured to, based on the gray region existing in the white balance-processed input image, increase edge resolution based on the second image corresponding to the edge direction determined by the edge detection unit and generate the third image by blending the first image and the edge resolution increased second image.

10. The apparatus for obtaining the image of claim 7 , wherein the selection unit is further configured to determine, by the gray detection unit, whether the gray region exists in the white balance-processed input image based on the edge strength of the reconstructed green signal being less than the preset value.

11. The apparatus for obtaining the image of claim 10 , wherein the selection unit is further configured to, based on the gray region existing in the white balance-processed input image, increase edge resolution based on the second image corresponding to the edge direction determined by the edge detection unit and generate the third image by blending the reconstructed green signal and the edge resolution increased second image.

12. The apparatus for obtaining the image of claim 1 , wherein the preset direction comprises a horizontal direction, a vertical direction, a 45° direction, and a 135° direction.

13. The apparatus for obtaining the image of claim 1 , wherein the direction image generation unit comprises:
a low-pass filter configured to generate a fifth image by performing low-frequency filtering on the white balance-processed input image; and
an operator configured to receive the white balance-processed input image and the fifth image, and subtract the fifth image from the white balance-processed input image.

14. The apparatus for obtaining the image of claim 13 , wherein the direction image generation unit further comprises a multiplier between the low-pass filter and the operator, the multiplier being configured to multiply the fifth image by a gain.

15. The apparatus for obtaining the image of claim 13 , wherein the direction image generation unit further comprises a high-pass filter configured to detect a limit resolution of the white balance-processed input image.

16. The apparatus for obtaining the image of claim 1 , wherein the edge detection unit comprises a low-pass filter configured to remove noise from the white balance-processed input image.

17. The apparatus for obtaining the image of claim 1 , wherein the selection unit is further configured to generate the third image based on field information corresponding to a chief ray angle of the image sensor.

18. The apparatus for obtaining the image of claim 1 , wherein the direction image generation unit, the gray detection unit, and the edge detection unit are disposed in parallel and share a memory with the demosaicing unit.

19. A method of obtaining an image in an apparatus configured to obtain an image, the apparatus comprising an image sensor comprising a color separation lens array configured to separate and concentrate incident light based on a color of the incident light, the method comprising:
demosaicing a green signal to reconstruct the green signal to have a full pixel resolution based on the input image;
generating a first image by sharpening the reconstructed green signal for each preset direction;
performing white balance processing on the input image;
generating a second image by removing a base band and extracting a detail band from the white balance-processed input image for each preset direction based on an unsharpening filtering algorithm;
detecting a gray region of the white balance-processed input image;
detecting an edge direction of the white balance-processed input image; and
generating a third image by blending the first image and the second image based on the detected gray region and the detected edge direction.

20. The method of obtaining the image of claim 19 , further comprising adjusting a red signal and a blue signal based on an amount of change of the green signal of the third image.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Jang Soongeun
• Kim Woo-Shik
• Azaria Haim
• Bitan Gal
• Avishai Igal
• Grechikhin Lena

Classifications

• H04N23/16

  Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction · CPC title

• H04N23/88

  for colour balance, e.g. white-balance circuits or colour temperature control · CPC title

• H04N23/683

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• G06T3/4015

  Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns · CPC title

• H04N23/843 Primary

  Demosaicing, e.g. interpolating colour pixel values · CPC title