Light-emitting diode (LED) flickering management (LFM) for spatially multiplexed image sensor

Patent metadata

Field	Value
Publication number	US-12464253-B2
Application number	US-202318526432-A
Country	US
Kind code	B2
Filing date	Dec 1, 2023
Priority date	Dec 1, 2023
Publication date	Nov 4, 2025
Grant date	Nov 4, 2025

Abstract

This application describes method and apparatus for LED Flickering Management (LFM) using spatially multiplexed image sensors. An example method includes configuring the sensor with diverse exposure settings to capture images of a scene with an LED light source. For each pixel position, multiple intensity values are collected across these images. An estimated linear regression model, incorporating an estimated Quantum Efficiency (QE) factor, is constructed using these values. The pixel intensities are then adjusted based on this estimated QE factor, aligning them with the regression model. This process results in modified images of the scene, effectively managing LED flickering for accurate image capture.

First claim

What is claimed is:

1 . A Light-Emitting Diode (LED) Flickering Management (LFM) method for an image sensor, comprising:
configuring the image sensor with a plurality of different exposure settings to capture a plurality of images of a scene comprising an LED light source; for each pixel position in the plurality of images:
obtaining a plurality of pixel intensity values at the pixel position across the plurality of images;
constructing an estimated linear regression model among the plurality of pixel intensity values, wherein the estimated linear regression model comprises an estimated Quantum Efficiency (QE) factor among the plurality of pixel intensity values;
adjusting the plurality of pixel intensity values at the pixel position across the plurality of images using the estimated QE factor such that the adjusted pixel intensity values are aligned with the estimated linear regression model; and
obtaining a plurality of adjusted images capturing the scene comprising the LED light source as a result of the adjusting the plurality of pixel intensity values at each pixel position in the plurality of images.

2 . The LFM method of claim 1 , wherein the plurality of different exposure settings correspond to linearly increased exposure times by a predetermined factor.

3 . The LFM method of claim 2 , wherein the plurality of different exposure settings comprise an initial setting with a base exposure time, and subsequent exposure settings have exposure times that are linearly increased from a previous exposure setting by the predetermined factor.

4 . The LFM method of claim 2 , wherein the adjusting the plurality of pixel intensity values using the estimated QE factor comprises:
identifying a first pixel intensity value from the plurality of pixel intensity values, wherein the first pixel intensity value corresponds to a first pixel and has a distance away from the estimated linear regression model that is greater than a threshold; obtaining, from the plurality of pixel intensity values, a second pixel intensity value corresponding to a second pixel; and adjusting the first pixel intensity value of the first pixel based on the second pixel intensity value of the second pixel, the estimated QE factor, and exposure times corresponding to the first pixel and the second pixel.

5 . The LFM method of claim 4 , wherein:
in response to an exposure time of the second pixel being greater than an exposure time of the first pixel by the predetermined factor, the adjusting the first pixel intensity value comprises: changing the first pixel intensity value to the second pixel intensity value divided by the estimated QE factor.

6 . The LFM method of claim 4 , wherein:
in response to an exposure time of the second pixel being shorter than an exposure time of the first pixel by the predetermined factor, the adjusting the first pixel intensity value comprises: changing the first pixel intensity value to the second pixel intensity value multiplied by the estimated QE factor.

7 . The LFM method of claim 1 , wherein the image sensor is a quad image sensor comprising a plurality of quaternions, each of the plurality of quaternions comprising a matrix of sensor pixels.

8 . The LFM method of claim 7 , wherein the configuring the image sensor to capture a plurality of images using a plurality of different exposure settings comprises:
configuring exposure time for sensor pixels in the quad image sensor such that the matrix of sensor pixels within each quaternion have different exposure times, and corresponding sensor pixels across the plurality of quaternions have a same exposure time.

9 . The LFM method of claim 8 , wherein the corresponding sensor pixels having the same exposure time across the plurality of quaternions capture one of the plurality of images.

10 . The LFM method of claim 1 , wherein the adjusting the plurality of pixel intensity values using the estimated QE factor comprises:
identifying a first pixel intensity value of the plurality of pixel intensity values that captures an ON cycle of the LED light source; identifying a second pixel intensity value of the plurality of pixel intensity values that captures an OFF cycle of the LED light source; and adjusting the second pixel intensity value based on the first pixel intensity value and the estimated QE factor.

11 . The LFM method of claim 1 , further comprising:
before adjusting the plurality of pixel intensity values using the estimated QE factor, fine-tuning the estimated QE factor in the linear regression model corresponding to a first pixel position based on QE factors corresponding to pixel positions surrounding the first pixel position.

12 . The LFM method of claim 1 , further comprising:
performing High Dynamic Range (HDR) fusion on the plurality of adjusted images to generate an HDR image of the scene comprising the LED light source.

13 . A system, comprising one or more processors and one or more non-transitory computer-readable memories coupled to the one or more processors and configured with instructions executable by the one or more processors to cause the system to perform operations comprising:
configuring an image sensor with a plurality of different exposure settings to capture a plurality of images of a scene comprising an LED light source; for each pixel position in the plurality of images:
obtaining a plurality of pixel intensity values at the pixel position across the plurality of images;
constructing an estimated linear regression model among the plurality of pixel intensity values, wherein the estimated linear regression model comprises an estimated Quantum Efficiency (QE) factor among the plurality of pixel intensity values;
adjusting the plurality of pixel intensity values at the pixel position across the plurality of images using the estimated QE factor such that the adjusted pixel intensity values are aligned with the estimated linear regression model; and
obtaining a plurality of adjusted images capturing the scene comprising the LED light source as a result of the adjusting the plurality of pixel intensity values at each pixel position in the plurality of images.

14 . The system of claim 13 , wherein the plurality of different exposure settings correspond to linearly increased exposure times by a predetermined factor.

15 . The system of claim 14 , wherein the adjusting the plurality of pixel intensity values using the estimated QE factor comprises:
identifying a first pixel intensity value from the plurality of pixel intensity values, wherein the first pixel intensity value corresponds to a first pixel and has a distance away from the estimated linear regression model that is greater than a threshold; obtaining, from the plurality of pixel intensity values, a second pixel intensity value corresponding to a second pixel; and adjusting the first pixel intensity value of the first pixel based on the second pixel intensity value of the second pixel, the estimated QE factor, and exposure times corresponding to the first pixel and the second pixel.

16 . The system of claim 15 , wherein in response to an exposure time of the second pixel being greater than an exposure time of the first pixel by the predetermined factor, the adjusting the first pixel intensity value comprises:
changing the first pixel intensity value to the second pixel intensity value divided by the estimated QE factor.

17 . The system of claim 13 , wherein the image sensor is a quad image sensor comprising a plurality of quaternions, each of the plurality of quaternions comprisi

Assignees

• Cista Sys Corp

Inventors

• Yang Shenglong
• Li Zhaojian

Classifications

• H04N23/73 Primary

  by influencing the exposure time · CPC title

• H04N23/743

  Bracketing, i.e. taking a series of images with varying exposure conditions · CPC title

• H04N23/72

  Combination of two or more compensation controls · CPC title

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N23/71

  Circuitry for evaluating the brightness variation · CPC title