High-dynamic range image sensor and image-capture method

Patent metadata

Field	Value
Publication number	US-11102422-B2
Application number	US-201916431887-A
Country	US
Kind code	B2
Filing date	Jun 5, 2019
Priority date	Jun 5, 2019
Publication date	Aug 24, 2021
Grant date	Aug 24, 2021

Abstract

A method for capturing a high-dynamic-range image includes: (i) storing a plurality of pixel values representing a first image captured by an image sensor that includes a pixel array, each pixel value having been generated by a respective pixel of a pixel subarray of the pixel array, each pixel being set to one of N1 first exposure values, N1≥1; (ii) determining an exposure-count N2 based on the plurality of pixel values; (iii) setting each pixel to one of a second plurality of exposure values, N2 in number, such that, for each of the second plurality of exposure values, at least one pixel is set to that exposure value, one of the second plurality of exposure values differing from each of the N1 first exposure values by more than a threshold value; and after setting (iv), capturing a second image with the image sensor.

First claim

What is claimed is:

1. A method for capturing a high-dynamic-range image comprising:
storing, in memory, a plurality of pixel values representing a first image captured by an image sensor that includes a pixel array, each of the plurality of pixel values having been generated by a respective one of a first plurality of pixels of a pixel subarray of the pixel array, each of the first plurality of pixels being set to one of N 1 first exposure values, N 1 ≥1;
reading the plurality of pixel values from the memory while skipping additional pixel values generated by an additional plurality of pixels, of the pixel subarray, each located between a respective pair of the first plurality of pixels;
determining an exposure-count N 2 based on the plurality of pixel values, N 2 being a positive integer;
setting each of the first plurality of pixels to one of a second plurality of exposure values, N 2 in number, such that, for each of the second plurality of exposure values, at least one pixel of the first plurality of pixels is set to that exposure value, one of the second plurality of exposure values differing from each of the N 1 first exposure values by more than a threshold value; and
after setting, capturing a second image with the image sensor.

2. The method of claim 1 , further comprising:
determining, based on the plurality of pixel values, each of the second plurality of exposure values.

3. The method of claim 1 , further comprising:
generating histogram data from the plurality of pixel values; and
determining each of the second plurality of exposure values based on the histogram data.

4. The method of claim 1 , the step of determining comprising:
applying an edge-detection method to the plurality of pixel values, N 2 exceeding N 1 when an edge is detected, and N 2 not exceeding N 1 when no edge is detected.

5. The method of claim 1 , the first plurality of pixels including a first sub-plurality of pixels configured to detect light in a same spectral range and generating a respective first sub-plurality of pixel values of the plurality of pixel values, the step of determining comprising:
computing a dispersion of the first sub-plurality of pixel values, N 2 exceeding N 1 when the dispersion exceeds a threshold value, and N 2 not exceeding N 1 when the dispersion is less than a threshold value.

6. The method of claim 1 , the step of determining comprising:
determining a number of saturated pixel-values, of the plurality of pixel values, N 2 exceeding N 1 when the number of saturated pixel-values exceeds a threshold value, and N 2 not exceeding N 1 when the number of saturated pixel-values is less than a threshold value.

7. The method of claim 1 , the first image being of a scene, the step of determining including:
comparing a previous image and the first image to determine a presence of either (i) object motion in the scene or (ii) image-sensor motion with respect to the scene, exposure-count N 2 being based at least in part on the presence of at least one of object motion and image-sensor motion, the previous image including the scene and captured by the image sensor prior to capture of the first image.

8. The method of claim 1 , the second plurality of exposure values including a first exposure value and a second exposure value, the step of setting further comprising setting a first percentage of the first plurality of pixels to the first exposure value and a second percentage of the first plurality of pixels to the second exposure value, the first percentage and the second percentage differing by less than twenty percentage points.

9. The method of claim 1 , N 1 ≥2, the N 1 exposure values including exposure values V 1 and V 2 , a first percentage of the first plurality of pixels having exposure value V 1 , a second percentage of the first plurality of pixels having exposure value V 2 , and in the step of determining, N 2 >N 1 , the N 2 exposure values including exposure values V 1 , V 2 , and V 3 , and further comprising:
setting a third percentage of the first plurality of pixels to exposure value V 3 , V 1 <V 3 <V 2 .

10. The method of claim 9 , the first percentage of the first plurality of pixels having generated a first sub-plurality of pixel values, the second percentage of the first plurality of pixels having generated a second sub-plurality of pixel values, and further comprising:
determining a noise metric indicative of a signal-to-noise level of the first sub-plurality of pixel values;
determining a saturation metric indicative of a degree of saturation of the second sub-plurality of pixel values;
determining an incongruence metric indicative of consistency of a first scene brightness, predicted by the first sub-plurality of pixel values, and a second scene brightness predicted by the second sub-plurality of pixel values; and
in the step of setting the third percentage, setting the third percentage of the first plurality of pixels to the exposure value V 3 when each of the noise metric, the saturation metric, and the incongruence metric exceeds a respective threshold value.

11. The method of claim 9 , the first percentage and the second percentage differing by less than twenty percentage points, and in the step of setting the third percentage, the third percentage being less than twenty percentage points.

12. The method of claim 1 , the step of setting further comprising:
setting a first percentage of the first plurality of pixels to a first exposure value and second percentage of the first plurality of pixels to a second exposure value, the first percentage and the second percentage differing by more than fifty percentage points.

13. The method of claim 1 , further comprising:
generating histogram data from the plurality of pixel values; and
when determining the exposure-count N 2 , determining the exposure-count N 2 based on the histogram data.

14. An image sensor comprising:
a pixel array including a pixel subarray that includes (i) a first plurality of pixels, and (ii) an additional plurality of pixels each located between a respective pair of the first plurality of pixels;
a processor coupled to the pixel array; and
a memory storing machine-readable instructions that, when executed by the processor, control the processor to:
read a plurality of pixel values from the memory while skipping additional pixel values generated by the additional plurality of pixels;
store, in the memory, the plurality of pixel values representing a first image captured by the image sensor, each of the plurality of pixel values having been generated by a respective one of the first plurality of pixels, each of the first plurality of pixels being set to one of N 1 first exposure values, N 1 ≥1;
determine an exposure-count N 2 based on the plurality of pixel values, N 2 being a positive integer;
set each of the first plurality of pixels to one of a second plurality of exposure values, N 2 in number, such that, for each of the second plurality of exposure values, at least one pixel of the first plurality of pixels is set to that exposure value, one of the second plurality of exposure values differing from each of the N 1 first exposure values by more than a threshold value;
and, after setting;
capture a second image with the image sensor.

15. The image sensor of claim 14 , the first plurality of pixels including a first sub-plurality of pixels configured to detect light in a same spectral range and generating a respective first sub-plurality of pixel values of the plurality of pixel values, the memory further storing machine-readable instructions that, when execut

Assignees

• Omnivision Tech Inc

Inventors

• Liu Chengming
• Dai Tiejun
• Mann Richard

Classifications

• H04N25/11

  Arrangement of colour filter arrays [CFA]; Filter mosaics · CPC title

• H04N25/59

  by controlling the amount of charge storable in the pixel, e.g. modification of the charge conversion ratio of the floating node capacitance · CPC title

• H04N25/583

  with different integration times · CPC title

• H04N25/771

  comprising storage means other than floating diffusion · CPC title

• H04N23/741 Primary

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title