Imaging systems and methods for generating motion-compensated high-dynamic-range images

Patent metadata

Field	Value
Publication number	US-9883125-B2
Application number	US-201414563887-A
Country	US
Kind code	B2
Filing date	Dec 8, 2014
Priority date	Oct 6, 2011
Publication date	Jan 30, 2018
Grant date	Jan 30, 2018

Abstract

Electronic devices may include image sensors. Image sensors may be used to capture images having rows of long-exposure image pixel values that are interleaved with rows of short-exposure image pixel values. The long-exposure and short-exposure values in each interleaved image frame may be interpolated to form interpolated values. A combined long-exposure image and a combined short-exposure image may be generated using the long-exposure and the short-exposure values from the interleaved image frames and the interpolated values from a selected one of the interleaved image frames. The combined long-exposure and short-exposure images may each include image pixel values from either of the interleaved image frames in a non-motion edge region and image pixel values based only on the image pixel values or the interpolated values from the selected one of the interleaved images in a motion or non-edge region. High-dynamic-range images may be generated using the combined long-exposure and short-exposure images.

First claim

What is claimed is:

1. A method of capturing images, comprising:
capturing a plurality of successive images, wherein a first one of the plurality of successive images has short-exposure image pixel values and long-exposure image pixel values in a first arrangement, and wherein a second one of the plurality of successive images has short-exposure image pixel values and long-exposure image pixel values in a second arrangement that is different than the first arrangement, wherein the short-exposure image pixel values and the long-exposure image pixel values in the first one of the plurality of successive images are captured by first and second respective sets of pixels, wherein the first set of pixels captures the long-exposure image pixel values in the second one of the plurality of successive images and the second set of pixels captures the short-exposure image pixel values in the second one of the plurality of successive images; and
processing the first one of the plurality of successive images and the second one of the plurality of successive images to generate a motion-compensated high dynamic range image.

2. The method defined in claim 1 , wherein processing the plurality of successive images further comprises:
with the first one of the plurality of successive images, generating at least a first interpolated image based on a selected one of the short-exposure image pixel values and the long-exposure image pixel values of the first image.

3. The method defined in claim 2 , wherein processing the plurality of successive images further comprises:
with the second one of the plurality of successive images, generating at least a second interpolated image based on a selected one of the short-exposure image pixel values and the long-exposure image pixel values of the second image.

4. The method defined in claim 3 , wherein processing the plurality of successive images further comprises:
generating a motion map based on the at least first interpolated image and the at least second interpolated image.

5. The method defined in claim 1 , wherein processing the plurality of successive images comprises:
generating an interpolated short-exposure image based on the short-exposure image pixel values of the first one of the plurality of successive images; and
generating an interpolated long-exposure image based on the long-exposure image pixel values of the first one of the plurality of successive images.

6. The method defined in claim 5 , wherein processing the plurality of successive images further comprises:
with the interpolated short-exposure image and the interpolated long-exposure image, generating an edge map.

7. The method defined in claim 5 , wherein processing the plurality of successive images further comprises:
with the first one of the plurality of successive images, the second one of the plurality of successive images, and the interpolated short-exposure image, generating a short-exposure image; and
with the first one of the plurality of successive images, the second one of the plurality of successive images, and the interpolated long-exposure image, generating a long-exposure image.

8. The method defined in claim 7 , wherein processing the plurality of successive images further comprises:
generating a reconstruction map; and
using the reconstruction map to generate the short-exposure image and the long-exposure image.

9. The method defined in claim 8 , wherein processing the plurality of successive images further comprises:
generating a motion map based on the first image and the second image;
generating a first edge map based on the first image;
generating a second edge map based on the second image; and
with the motion map, the first edge map, and the second edge map, generating the reconstruction map.

10. The method defined in claim 7 , wherein processing the plurality of successive images further comprises:
with the short-exposure image and the long-exposure image, generating the motion-compensated high dynamic range image.

11. The method defined in claim 1 , wherein the first one of the plurality of successive images has intermediate-exposure image pixel values in the first arrangement, and wherein the second one of the plurality of successive images has intermediate-exposure pixel values in the second arrangement.

12. A method of generating a motion-compensated high dynamic range image using processing circuitry and an image sensor having pixels, the method comprising:
with the image sensor, capturing a first image that includes a first set of pixels having image pixel values for a first exposure time and a second set of pixels having image pixel values for a second exposure time that is different than the first exposure time;
with the image sensor, capturing a second image that includes the first set of pixels having image pixel values for the second exposure time and the second set of pixels having image pixel values for the first exposure time; and
with the processing circuitry, generating the motion-compensated high dynamic range image by processing the first and second images.

13. The method defined in claim 12 , further comprising:
selecting one of the first and second images as a reference image.

14. The method defined in claim 13 , further comprising:
identifying a motion region based on the first image and the second image.

15. The method defined in claim 14 , further comprising:
identifying at least one edge region in the first image; and
identifying at least one edge region in the second image.

16. The method defined in claim 15 , wherein the reference image is selected based on the motion region, the at least one edge region in the first image, and the at least one edge region in the second image.

17. The method defined in claim 13 , wherein the motion-compensated high dynamic range image is generated from the reference image.

18. The method defined in claim 13 , wherein capturing the first image comprises capturing the first image that includes a third set of pixels having image pixel values for a third exposure time that is different than the first and second exposure times.

19. A system, comprising:
a central processing unit;
memory;
input-output circuitry;
an imaging device, wherein the imaging device comprises:
an image sensor configured to capture a first image and a second image, wherein the first image is captured using a first set of pixels to generate first short-exposure pixel values and a second set of pixels to capture first long-exposure pixel values, and wherein the second image is captured using the first set of pixels to generate second long-exposure pixel values and the second set of pixels to capture second short-exposure pixel values; and
processing circuitry configured to process the first and second images to generate a motion-compensated high dynamic range output image.

20. The system of claim 19 , wherein the first image is further captured using a third set of pixels to generate third short-exposure pixel values, wherein the second image is further captured using the third set of pixels to generate third long-exposure pixel values, and wherein the processing circuitry is further configured to generate interpolated short-exposure pixel values for the second set of pixels based on the first and third short-exposure pixel values and to generate interpolated long-exposure pixel values based on the second and third long-exposure pixel values.

21. The system of claim 20 , wherein the first image is further captur

Assignees

• Semiconductor Components Ind Llc

Inventors

• Li Huai Dong
• Lin Peng

Classifications

• H04N25/583

  with different integration times · CPC title

• H04N23/6811 Primary

  based on the image signal · CPC title

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N25/589 Primary

  with different integration times, e.g. short and long exposures · CPC title

• H04N23/73

  by influencing the exposure time · CPC title