Image processing device for noise reduction using dual conversion gain and operation method thereof

Patent metadata

Field	Value
Publication number	US-11722795-B2
Application number	US-202217670793-A
Country	US
Kind code	B2
Filing date	Feb 14, 2022
Priority date	Jun 25, 2021
Publication date	Aug 8, 2023
Grant date	Aug 8, 2023

Abstract

Disclosed is a device for noise reduction using dual conversion gain, which includes an image sensor including a pixel array, the pixel array configured to generate a first pixel signal corresponding to a first conversion gain and a second pixel signal corresponding to a second conversion gain from pixels sharing a floating diffusion region and the image sensor configured to generate first image data and second image data based on the first pixel signal and the second pixel signal, and an image signal processor that generates an output image based on the first image data and the second image data. The image signal processor includes a normalization circuit that normalizes the first image data based on a dynamic range of the second image data to generate third image data, and a blending circuit that generates the output image based on the second image data and the third image data.

First claim

What is claimed is:

1. An image processing device comprising:
an image sensor including a pixel array, the pixel array being configured to
generate a first pixel signal corresponding to a first conversion gain based on receiving light reflected from an object, and
generate a second pixel signal based on receiving light reflected from the object corresponding to a second conversion gain from pixels sharing a floating diffusion region, and
the image sensor being configured to generate first image data and second image data based on the first pixel signal and the second pixel signal; and

an image signal processor configured to generate an output image based on the first image data and the second image data, the image signal processor including
a normalization circuit configured to normalize the first image data based on a dynamic range of the second image data to generate third image data; and
a blending circuit configured to generate the output image based on the second image data and the third image data in response to an ambient illuminance of the object.

2. The image processing device of claim 1 , wherein
the first pixel signal corresponds to charges stored at the floating diffusion region having a first capacitance value, and
the second pixel signal corresponds to charges stored at the floating diffusion region having a second capacitance value.

3. The image processing device of claim 2 , wherein the pixel array includes a capacitor connected with the floating diffusion region to obtain the second capacitance value.

4. The image processing device of claim 1 , wherein the blending circuit is configured to
assign a first weight to the third image data and
assign a second weight to the second image data.

5. The image processing device of claim 4 , wherein each of the first weight and the second weight has a value between “0” and “1”, and a sum of the first weight and the second weight is “1”.

6. The image processing device of claim 1 , wherein the pixels of the pixel array include first pixels controlled based on a first exposure time and second pixels controlled based on a second exposure time, and
wherein, based on a number of the first pixels and a number of the second pixels being equal, the blending circuit is configured to generate the output image based on the second image data and the third image data.

7. An electronic device comprising:
a lens configured to receive a light reflected from an object;
an image sensor including a pixel array, the pixel array configured to
generate a first pixel signal corresponding to a first conversion gain and a second pixel signal corresponding to a second conversion gain from pixels sharing a floating diffusion region, and
the image sensor configured to receive a light from the lens and generate first image data and second image data based on the first pixel signal and the second pixel signal;

an image signal processor configured to generate an output image based on the first image data or the second image data; and
a main processor configured to generate a video stream based on the output image,
the image signal processor including
a normalization circuit configured to normalize the first image data based on a dynamic range of the second image data to generate third image data; and
a blending circuit configured to generate the output image based on the second image data and the third image data in response to an ambient illuminance of the object.

8. The electronic device of claim 7 , wherein
the first pixel signal corresponds to charges stored at the floating diffusion region having a first capacitance value, and
the second pixel signal corresponds to charges stored at the floating diffusion region having a second capacitance value.

9. The electronic device of claim 8 , wherein the pixel array includes a capacitor connected with the floating diffusion region to obtain the second capacitance value.

10. The electronic device of claim 7 , wherein the blending circuit is configured to assign a first weight to the third image data and assign a second weight to the second image data.

11. The electronic device of claim 10 , wherein each of the first weight and the second weight has a value between “0” and “1”, and a sum of the first weight and the second weight is “1”.

12. The electronic device of claim 7 , wherein the main processor is configured to determine to activate an operation of the blending circuit based on receiving a user input.

13. The electronic device of claim 7 , wherein the main processor is configured to output the video stream to a display device as a preview image in which a noise is reduced.

14. The electronic device of claim 7 , wherein
the pixels of the pixel array include first pixels controlled based on a first exposure time and second pixels controlled based on a second exposure time, and
based on a number of the first pixels and a number of the second pixels being equal, the blending circuit is configured to generate the output image based on the second image data and the third image data.

15. An operation method of a device for noise reduction, the method comprising:
generating first image data corresponding to a first conversion gain and second image data corresponding to a second conversion gain;
normalizing the first image data based on a dynamic range of the second image data to generate third image data;
generating an output image based on the first image data or the second image data;
determining to perform the generating of the third image data and the generating of the output image based on an ambient illuminance of an object corresponding to the first image data and the second image data; and
generating a video stream based on the output image.

16. The method of claim 15 , wherein the first conversion gain is smaller than the second conversion gain.

17. The method of claim 15 , wherein the generating of the output image includes:
assigning a first weight and a second weight to the third image data and the second image data, respectively.

18. The method of claim 17 , wherein each of the first weight and the second weight has a value between “0” and “1”, and a sum of the first weight and the second weight is “1”.

19. The method of claim 15 , further comprising:
outputting the video stream to a display device as a preview image in which a noise is reduced.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Daeyoung
• Kim Kundong
• Kim Jongwoo
• Lee Suyoung

Classifications

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T5/70 Primary

  Denoising; Smoothing · CPC title

• H04N25/60

  Noise processing, e.g. detecting, correcting, reducing or removing noise · CPC title

• H04N25/59 Primary

  by controlling the amount of charge storable in the pixel, e.g. modification of the charge conversion ratio of the floating node capacitance · CPC title

• H04N25/587

  acquired sequentially, e.g. using the combination of odd and even image fields · CPC title