Pulsed light imaging using rolling shutter image sensors

Patent metadata

Field	Value
Publication number	US-11528434-B1
Application number	US-202117562781-A
Country	US
Kind code	B1
Filing date	Dec 27, 2021
Priority date	Feb 9, 2021
Publication date	Dec 13, 2022
Grant date	Dec 13, 2022

Abstract

An imaging system and method for imaging a scene using a rolling shutter are described. In an embodiment, the method includes illuminating a scene with first and second illumination light; generating frame signals with a photodetector comprising a plurality of pixels arranged in a plurality of rows, wherein the frame signals are based on light received from the scene with sequentially integrated rows of pixels of the plurality of rows, and wherein a frame signal includes signals from pixels of each of the plurality of rows; and generating images of the scene based on an intensity of the frame signals and the proportion of the first illumination light and the second illumination light emitted onto the scene during the first and second frames, wherein a proportion of the first illumination light and the second illumination light in a first frame is different than in a second frame.

First claim

What is claimed is:

1. A system comprising:
a light engine configured to illuminate a scene, the light engine comprising:
a first light source configured emit first illumination light; and
a second light source configured to emit second illumination light;

a photodetector positioned to receive light from the scene, the photodetector comprising a plurality of pixels responsive to light and arranged in a plurality of rows; and
a controller operatively coupled to the light engine and the photodetector, the controller including logic that, when executed by the controller, causes the system to perform operations including:
illuminating the scene with the first illumination light from the first light source and the second illumination light from the second light source;
generating frame signals with the photodetector based on light received from the scene with sequentially integrated rows of pixels of the plurality of rows, wherein a frame signal includes signals from pixels of each of the plurality of rows; and
generating images of the scene based on an intensity of the frame signals and the proportion of the first illumination light and the second illumination light emitted onto the scene during the first and second frames, wherein a proportion of the first illumination light and the second illumination light in a first frame is different than in a second frame.

2. The system of claim 1 , wherein illuminating the scene with the first illumination light and the second illumination light includes:
emitting a plurality of pulses of the first illumination light with the first light source; and
emitting a plurality of pulses of the second illumination light with the second light source interleaved between the plurality of pulses of the first illumination light,
wherein a light pulse of the plurality of first illumination light is captured by two sequential frames.

3. The system of claim 2 , wherein a light pulse of the plurality of the second illumination light is captured by two sequential frames.

4. The system of claim 2 , wherein a pulse of the plurality of pulses of the first illumination light has a duration different than a duration of the plurality of pulses of the second illumination light.

5. The system of claim 1 , wherein generating frame signals with the photodetector based on light received from the scene with sequentially integrated rows of pixels of the plurality of rows includes:
generating first signals with a first row of pixels of the photodetector based on light received from the scene during a first integration time; and
generating second frame signals with a second row of pixels of the photodetector based on light received from the scene during a second integration time,
wherein the second row of pixels is immediately adjacent to the first row of pixels and the second integration time is after the first integration time.

6. The system of claim 1 , wherein a frame signal from a pixel i,j of the photodetector in a frame N is according to equation (1):
S N ( i,j )=frac N,L1 ( i,j ) S L1 ( i,j )+frac N,L2 ( i,j ) S L2 ( i,j )  equation (1),

wherein—
frac N,L1 (i,j) is a fraction of an integration time that pixel i,j was exposed to the first illumination light during frame N,
frac N,L2 (i,j) is a fraction of the integration time that pixel i,j was exposed to the second illumination light during frame N,
S L1 (i,j) is a signal generated by pixel i,j if pixel i,j was illuminated by the first illumination light during all of the integration time, and
S L2 (i,j) a signal generated by pixel i,j if pixel i,j was illuminated by the second illumination light during all of the integration time.

7. The system of claim 1 , wherein a frame signal from a frame N+1 immediately following frame N is according to equation (2)
S N+1 ( i,j )=frac N+1,L1 ( i,j ) S L1 ( i,j )+frac N+1,L2 ( i,j ) S L2 ( i,j )  equation (2),

wherein—
frac N+1,L1 (i,j) is a fraction of an integration time pixel i,j was exposed to the first illumination light during frame N+1, and
frac N+1,L2 (i,j) is a fraction of the integration time pixel i,j was exposed to the second illumination light during frame N+1.

8. The system of claim 1 , further comprising a filter positioned to filter first and second illumination light received from the scene and provide filtered light to the photodetector,
wherein the filter comprises:
a plurality of first filter pixels positioned to filter illumination light incident on a first set of pixels of the photodetector and configured to transmit light in a first filtered wavelength range; and
a plurality of second filter pixels positioned to filter illumination light incident on a second set of pixels of the photodetector and configured to transmit light in a second filtered wavelength range, wherein the second filtered wavelength range is different than the first filtered wavelength range.

9. The system of claim 1 , further comprising an endoscope coupled to and configured to receive from the light engine and shaped to illuminate the scene with the illumination light.

10. The system of claim 1 , wherein the first illumination light is in a first wavelength range, and wherein the second illumination light is in a second wavelength range different from the first wavelength range.

11. The system of claim 1 , wherein the second light source is configured to emit structured light, and
wherein the controller includes logic that, when executed by the controller, causes the system to perform operations including:
determining a distance between features of the scene based on structured light received from the scene by the photodetector.

12. A non-transitory, machine-readable storage medium having instructions stored thereon, which when executed by a processing system, cause the processing system to perform a method comprising:
illuminating a scene with first illumination light from a first light source and second illumination light from a second light source;
generating frame signals with a photodetector comprising a plurality of pixels responsive to light and arranged in a plurality of rows, wherein the frame signals are based on light received from the scene with sequentially integrated rows of pixels of the plurality of rows, and wherein a frame signal includes signals from pixels of each of the plurality of rows; and
generating images of the scene based on an intensity of the frame signals and the proportion of the first illumination light and the second illumination light emitted onto the scene during the first and second frames, wherein a proportion of the first illumination light and the second illumination light in a first frame is different than in a second frame.

13. The non-transitory, machine-readable storage medium of claim 12 , wherein illuminating the scene with the first illumination light and the second illumination light includes:
emitting a plurality of pulses of the first illumination light with the first light source; and
emitting a plurality of pulses of the second illumination light with the second light source interleaved between the plurality of pulses of the first illumination light,
wherein a light pulse of the plurality of first illumination light is captured by two sequential frames.

14. The non-transitory, machine-readable storage medium of claim 13 , wherein a light pulse of the plurality of the second illumination light is captured by two sequential frames.

15. The non-transitory, machine-readable storage medium of claim 13 , wherein a pulse of the plurality of pulses of the first illumination light has a duration di

Assignees

• Verily Life Sciences Llc

Inventors

• Bouvier Aurelien
• Verma Nishant
• Polans James

Classifications

• H04N25/531 Primary

  by controlling rolling shutters in CMOS SSIS · CPC title

• H04N23/56

  provided with illuminating means · CPC title

• H04N25/134

  based on three different wavelength filter elements · CPC title

• H04N23/555 Primary

  for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes · CPC title

• H04N5/3532 Primary

  Electricity · mapped topic