LED flicker reduction for high dynamic range imaging

Patent metadata

Field	Value
Publication number	US-11019277-B2
Application number	US-201916543760-A
Country	US
Kind code	B2
Filing date	Aug 19, 2019
Priority date	Nov 30, 2018
Publication date	May 25, 2021
Grant date	May 25, 2021

Abstract

A system for capturing a high dynamic range (HDR) image comprises an image sensor comprising a split pixel including a first pixel having higher effective gain and a second pixel having lower effective gain. The second pixels exposed with a capture window capture at least a pulse emitted by a light emitting diode (LED) controlled by a pulse width modulation. A first HDR image is produced by a combination including an image produced by the second pixels, and images produced by multiple exposures of the first pixels. A weight map of LED flicker correction is generated from the difference of the image produced by second pixels and the images produced by the first pixels, and the flicker areas in the first HDR image are corrected with the weight map and the image from the second pixels.

First claim

What is claimed is:

1. A system for capturing a high dynamic range (HDR) image comprising:
an image sensor comprising a split pixel including a first pixel having higher effective gain and a second pixel having lower effective gain;
wherein the first pixel is multiple exposed including a first exposure and a second exposure within a first pixel frame;
wherein the second pixel is exposed with a capture window within a second pixel frame, the capture window captures at least one pulse emitted by a light emitting diode (LED) controlled by a pulse width modulation;
wherein a first HDR image is produced by a combination including an image produced by the second pixel, a first image produced by the first exposure of the first pixel, and a second image produced by the second exposure of the first pixel; and
a processor, wherein a final flicker corrected HDR image is outputted from the processor, and wherein the final flicker corrected HDR is produced as follows:
HDR 2( x,y )=[ S ( x,y )× W ( x,y )]+[ HDR 1( x,y )×(1− W ( x,y ))],

wherein:
0≤W(x,y)≤1,
wherein HDR2(x,y) is the final flicker corrected HDR image, S(x,y) is the image produced by the second pixel, HDR1(x,y) is the first HDR image, and W(x,y) is a total weight, which is a function of individual weights and the individual weight is a function of an absolute value of difference between the image produced by the second pixel, and an image produced by the first pixel, and a noise model based threshold.

2. The system of claim 1 , wherein the first exposure has a first exposure time and the second exposure has a second exposure time, the first exposure time is longer than the second exposure time.

3. The system of claim 1 , wherein the first pixel frame is same as the second pixel frame.

4. The system of claim 1 , wherein the first pixel is multiple exposed including more than two exposures.

5. The system of claim 1 , wherein the second pixel has smaller area than the first pixel.

6. The system of claim 1 , wherein the second pixel is covered by an attenuator.

7. The system of claim 1 , wherein an effective gain of the first pixel and an effective gain of the second pixel are controlled by an electric circuit designed such that the effective gain of the second pixel is lower than the effective gain of the first pixel.

8. A method for capturing a high dynamic range (HDR) image comprising:
capturing an image by lower effective gain pixels of split pixels of an image sensor, wherein a capture window captures at least one pulse emitted by a light emitting diode (LED) controlled by a pulse width modulation, wherein the capture window is within a frame of the lower effective gain pixel;
capturing a first image by higher effective gain pixels of the split pixels of the image sensor with a first exposure;
capturing a second image by the higher effective gain pixels of the split pixels of the image sensor with a second exposure;
wherein the first exposure and the second exposure are within a frame of the higher effective gain pixel;
combining at least the image captured by the lower effective gain pixels of slit pixels of the image sensor, the first image and the second image captured by the higher effective gain pixels of the split pixels of the image sensor to produce a first HDR image; and producing a final flicker corrected HDR image as follows:
HDR 2( x,y )=[ S ( x,y )× W ( x,y )]+[ HDR 1( x,y )×(1−( x,y ))],

wherein:
0≤W(x,y)≤1,
wherein HDR2 x,y) is the final flicker corrected HDR image, S(x,y) is the image captured by the second pixel, HDR1(x,y) is the first HDR image, and W(x,y) is a total weight, which is a function of individual weights, and the individual weight is a function of an absolute value of difference between the image produced by the second pixel and image produced by the first pixel, and a noise model based thresholds.

9. The system, of claim 8 , wherein the first exposure has a first exposure time and the second exposure has a second exposure time, the first exposure time is longer than the second exposure time.

10. The method of claim 8 , wherein the frame of the higher effective gain pixel is same as the frame of the lower effective gain pixel.

11. A camera for capturing a high dynamic range (HDR) image comprising:
an image sensor comprising a plurality of split pixels, each split pixel includes a first pixel having higher effective gain and a second pixel having lower effective gain;
a lens system forming an image of an field of view including a light emitting diode (LED) on the image sensor;
wherein the first pixel is multiple exposed including a first exposure and a second exposure within a first pixel frame;
wherein the second pixel is exposed with a capture window within a second pixel frame, the capture window captures at least one pulse emitted by the LED controlled by a pulse width modulation;
wherein a first HDR image is produced by a combination including an image produced by the second pixel, a first image produced by the first exposure of the first pixel, and a second image produced by the second exposure of the first pixel; and
a processor, wherein a final flicker corrected HDR image is outputted from the processor, and wherein the final flicker corrected HDR is produced as follows:
HDR 2( x,y )=[ S ( x,y )× W ( x,y )]+[ HDR 1( x,y )× W (1− W ( x,y ))],

wherein:
0≤W(x,y)≤1,
wherein HDR2(x,y) is the final flicker corrected HDR image, S(x,y) is the image produced by the second pixel, HDR1(x,y) is the first HDR image, and W(x,y) is a total weight, which is a function of individual weights, and the individual weight is a function of an absolute value of difference between the image produced by the second pixel and an image produced by the first pixel, and a noise model based thresholds.

12. The camera of claim 11 , wherein the first exposure has a first exposure time and the second exposure has a second exposure time, the first exposure time is longer than the second exposure time.

13. The camera of claim 11 , wherein the first pixel frame is same as the second pixel frame.

14. The camera of claim 11 , wherein the second pixel has smaller area than the first pixel.

15. The camera of claim 11 , wherein the second pixel is covered by an attenuator.

16. The camera of claim 11 , wherein an effective gain of the first pixel and an effective gain of the second pixel are controlled by an electric circuit designed such that the effective gain of the second pixel is lower than the effective gain of the first pixel.

17. The camera of claim 11 , wherein the first pixel is multiple exposed including more than two exposures.

Assignees

• Jiang Liangtao
• Lin Peng
• Omnivision Tech Inc

Inventors

• Jiang Liangtao
• Lin Peng

Classifications

• H04N25/585

  with pixels having different sensitivities within the sensor, e.g. fast or slow pixels or pixels having different sizes · CPC title

• H04N23/745 Primary

  Detection of flicker frequency or suppression of flicker wherein the flicker is caused by illumination, e.g. due to fluorescent tube illumination or pulsed LED illumination · CPC title

• H04N23/72

  Combination of two or more compensation controls · CPC title

• H04N23/73

  by influencing the exposure time · CPC title

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title