Electronic device comprising image sensor and method of operating same for processing single and multi-frame high dynamic range images

Patent metadata

Field	Value
Publication number	US-12477241-B2
Application number	US-202318195547-A
Country	US
Kind code	B2
Filing date	May 10, 2023
Priority date	Nov 11, 2020
Publication date	Nov 18, 2025
Grant date	Nov 18, 2025

Abstract

An electronic device including an image sensor and a processor may be provided. The image sensor may provide, in a first mode, to the at least one processor, first image data may be obtained by outputting data input to a unit pixel of the image sensor with a first conversion gain, the first image data may have a first number of bits. The image sensor may provide, in a second mode, to the at least one processor, second image data may be obtained by outputting data input to the unit pixel of the image sensor with the first conversion gain and a second conversion gain. The second conversion gain may be lower than the first conversion gain. The second image data may have a second number of bits which may be larger than the first number of bits.

First claim

What is claimed is:

1 . An electronic device comprising:
an image sensor; and at least one processor electrically connected to the image sensor, wherein the image sensor is configured to: activate a first mode or a second mode based on at least one of a change in illumination, subject movement, or a user setting; provide first image data having a first number of bits to the at least one processor, wherein the image sensor is in the first mode, and wherein the first image data is obtained by outputting data input to a unit pixel of the image sensor at a first conversion gain; and provide second image data having a second number of bits to the at least one processor, wherein the image sensor is in a single-frame high-dynamic range mode of the second mode, wherein the second mode comprises the single-frame high-dynamic range mode or multi-frame/single-frame high-dynamic range mode, wherein the second number of bits is larger than the first number of bits, and wherein the second image data is obtained by outputting the data input to the unit pixel of the image sensor at the first conversion gain and a second conversion gain lower than the first conversion gain, wherein the image sensor is further configured to add an embedded header to the first image data and the second image data and provide the embedded header to the at least one processor, wherein the embedded header comprises information about a number of bits of image data output by the image sensor.

2 . The electronic device of claim 1 , wherein the first mode is a low-illuminance mode, and
wherein the image sensor is further configured, in the first mode, to obtain the first image data by outputting the data input to the unit pixel of the image sensor two or more times at the first conversion gain and calculating an average conversion gain.

3 . The electronic device of claim 1 , wherein the first mode is a normal mode, and
wherein the image sensor is further configured to, in the first mode, obtain the first image data by outputting the data input to the unit pixel of the image sensor at the first conversion gain or outputting the data input to the unit pixel of the image sensor at the second conversion gain.

4 . The electronic device of claim 1 , wherein the image sensor is further configured to provide the at least one processor, in the first mode, with third image data having the second number of bits, wherein the third image data is obtained by adding dummy data having a third number of bits to the first image data having the first number of bits.

5 . The electronic device of claim 4 , wherein the first number of bits is 10, the second number of bits is 12, and the third number of bits is 2.

6 . The electronic device of claim 1 , wherein the image sensor is further configured to provide the at least one processor, in the first mode, with the first image data having the first number of bits and fourth image data having the first number of bits, wherein the first image data is obtained by outputting the data input to the unit pixel of the image sensor during a first exposure time at the first conversion gain, and wherein the fourth image data is obtained by outputting the data input to the unit pixel of the image sensor during a second exposure time at the second conversion gain.

7 . The electronic device of claim 6 , wherein the first mode is a multi-frame high-dynamic range (MF HDR) mode, and
wherein the first exposure time is greater than the second exposure time.

8 . The electronic device of claim 6 , wherein the image sensor is further configured to provide the at least one processor, in the first mode, with third image data having the second number of bits and fifth image data having the second number of bits, wherein the third image data is obtained by adding dummy data having a third number of bits to the first image data, and wherein the fifth image data is obtained by adding the dummy data having the third number of bits to the fourth image data.

9 . The electronic device of claim 1 , wherein the image sensor is further configured to provide the at least one processor, in the second mode, with the second image data having the second number of bits and sixth image data having the second number of bits, wherein the second image data is obtained by outputting the data input to the unit pixel of the image sensor during a first exposure time at the first conversion gain and the second conversion gain, and wherein the sixth image data is obtained by outputting the data input to the unit pixel of the image sensor during a second exposure time at the first conversion gain and the second conversion gain.

10 . The electronic device of claim 1 , wherein the embedded header comprises information about a type of conversion gain at which the data input to the unit pixel of the image sensor is output.

11 . A method of operating an electronic device, the method being performed by an image sensor included in the electronic device, the method including:
activating a first mode or a second mode base dion at least one of change in illumination, subject movement, or a user setting: obtaining, when the image sensor is in the first mode, first image data having a first number of bits by outputting data input to a unit pixel of the image sensor at a first conversion gain; obtaining, when the image sensor is in a single-frame high-dynamic range mode of the second mode, wherein the second mode comprises the single-frame high-dynamic range mode or multi-frame/single-frame high-dynamic range mode, second image data having a second number of bits by outputting the data input to the unit pixel of the image sensor at the first conversion gain and a second conversion gain, the second number of bits being larger than the first number of bits, and the second conversion gain is lower than the first conversion gain; and adding an embedded header to the first image data and the second image data and providing the first image data and the second image data to at least one processor included in the electronic device, wherein the embedded header comprises information about a number of bits of image data output by the image sensor.

12 . The method of claim 11 , wherein the image sensor is in the first mode, and the method further comprises:
obtaining, by the image sensor, third image data having the second number of bits by adding dummy data having a third number of bits to the first image data having the first number of bits; and providing, by the image sensor, the third image data to the at least one processor.

13 . The method of claim 11 , wherein the image sensor is in the second mode, and the method further comprises:
obtaining, by the image sensor, the second image data having the second number of bits by outputting the data input to the unit pixel of the image sensor during a first exposure time at the first conversion gain and the second conversion gain; obtaining, by the image sensor, sixth image data having the second number of bits by outputting the data input to the unit pixel of the image sensor during a second exposure time at the first conversion gain and the second conversion gain; and providing, by the image sensor, the second image data and the sixth image data to the at least one processor.

14 . An electronic device comprising:
an image sensor; and at least one processor electrically connected to the image sensor, wherein the image sensor is configured to:
provide the at least one processor with image data obtained by reading out data input to a unit pixel of the image sensor at a first conversion gain and a second conversion gain lower than t

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Dongsoo
• Kang Hwayong
• Kang Kawang
• Park Jaehyoung
• Song Byeongjoo
• Usui Takafumi
• Jeong Jaeoh
• Jo Hyeoncheol

Classifications

• H04N25/618

  for random or high-frequency noise · CPC title

• H04N25/616

  involving a correlated sampling function, e.g. correlated double sampling [CDS] or triple sampling · CPC title

• H04N25/59 Primary

  by controlling the amount of charge storable in the pixel, e.g. modification of the charge conversion ratio of the floating node capacitance · CPC title

• H04N25/587

  acquired sequentially, e.g. using the combination of odd and even image fields · CPC title

• H04N25/585

  with pixels having different sensitivities within the sensor, e.g. fast or slow pixels or pixels having different sizes · CPC title