Single image sensor for capturing mixed structured-light images and regular images

Patent metadata

Field	Value
Publication number	US-11102428-B2
Application number	US-201916563458-A
Country	US
Kind code	B2
Filing date	Sep 6, 2019
Priority date	Oct 16, 2015
Publication date	Aug 24, 2021
Grant date	Aug 24, 2021

Abstract

An integrated image sensor for capturing a mixed structured-light image and regular image using an integrated image sensor are disclosed. The integrated image sensor comprises a pixel array, one or more output circuits, one or more analog-to-digital converters, and one or more timing and control circuits. The timing and control circuits are arranged to perform a set of actions including capturing a regular image and a structured-light image. According to the present invention, the structured-light image captured before or after the regular image is used to derive depth or shape information for the regular image. An endoscope based on the above integrated image sensor is also disclosed. The endoscope may comprises a capsule housing adapted to be swallowed, where the components of integrated image sensor, a structured light source and anon-structured light source are enclosed and sealed in the capsule housing.

First claim

The invention claimed is:

1. An integrated image sensor, comprising:
a pixel array being responsive to light energy received by the pixel array to produce pixel signals having a voltage level depending on the light energy received by the pixel array;
one or more output circuits coupled to the pixel array to access the pixel signals produced by the pixel array;
one or more analog-to-digital converters having a first dynamic range and a second dynamic range;
one or more timing and control circuits coupled to the pixel array, said one or more output circuits, said one or more analog-to-digital converters or a combination thereof, wherein said one or more timing and control circuits are arranged to:
capture, by the pixel array, a structured-light image formed on a common image plane during a first frame period by applying first reset signals to the pixel array to reset rows of pixels of the pixel array, exposing the rows of pixels of the pixel array to first illumination from a structured light source to cause first analog signals from the rows of pixels and converting the first analog signals from the rows of pixels of the pixel array into first digital outputs for the structured-light image using one or more analog-to-digital converters;
capture, by the pixel array, a first regular image formed on the common image plane during a second frame period by applying second reset signals to the pixel array to reset the rows of pixels of the pixel array, exposing the rows of pixels to second illumination from a non-structured light source to cause second analog signals from the rows of pixels, and converting the second analog signals from the rows of pixels into second digital outputs for the first regular image using said one or more analog-to-digital converters;
capture, by the pixel array, a second regular image formed on the common image plane during a third frame period by applying third reset signals to the pixel array to reset the rows of pixels of the pixel array, exposing the rows of pixels to the second illumination from the non-structured light source to cause third analog signals from the rows of pixels, and converting the third analog signals from the rows of pixels into third digital outputs for the second regular image using said one or more analog-to-digital converters;
combine the second digital outputs and the third digital outputs to form a combined regular image; and
wherein the structured-light image is captured between the first regular image and the second regular image to derive depth or shape information for the combined regular image.

2. The integrated image sensor of claim 1 , wherein said one or more analog-to-digital converters correspond to a configurable analog-to-digital converter to provide both the first dynamic range and the second dynamic range.

3. The integrated image sensor of claim 1 , wherein said one or more analog-to-digital converters correspond to two analog-to-digital converters with two different resolutions to provide both the first dynamic range and the second dynamic range.

4. The integrated image sensor of claim 1 , wherein said one or more timing and control circuits are further arranged to cause the structured-light image to be captured at lower spatial resolution than the first regular image and the second regular image.

5. A camera system, comprising:
a structured light source;
a non-structured light source; and
an integrated image sensor, comprising:
a pixel array being responsive to light energy received by the pixel array to produce pixel signals having a voltage level depending on the light energy received by the pixel array;
one or more output circuits coupled to the pixel array to access the pixel signals produced by the pixel array;
one or more analog-to-digital converters having a first dynamic range and a second dynamic range;
one or more timing and control circuits coupled to the pixel array, said one or more output circuits, said one or more analog-to-digital converters or a combination thereof, wherein said one or more timing and control circuits are arranged to:
capture, by the pixel array, a structured-light image formed on a common image plane during a first frame period by applying first reset signals to the pixel array to reset rows of pixels of the pixel array, exposing the rows of pixels of the pixel array to first illumination from the structured light source to cause first analog signals from the rows of pixels and converting the first analog signals from the rows of pixels of the pixel array into first digital outputs for the structured-light image using one or more analog-to-digital converters;
capture, by the pixel array, a first regular image formed on the common image plane during a second frame period by applying second reset signals to the pixel array to reset the rows of pixels of the pixel array, exposing the rows of pixels to second illumination from the non-structured light source to cause second analog signals from the rows of pixels, and converting the second analog signals from the rows of pixels into second digital outputs for the first regular image using said one or more analog-to-digital converters;
capture, by the pixel array, a second regular image formed on the common image plane during a third frame period by applying third reset signals to the pixel array to reset the rows of pixels of the pixel array, exposing the rows of pixels to the second illumination from the non-structured light source to cause third analog signals from the rows of pixels, and converting the third analog signals from the rows of pixels into third digital outputs for the second regular image using said one or more analog-to-digital converters;
combine the second digital outputs and the third digital outputs to form a combined regular image; and
wherein the structured-light image is captured between the first regular image and the second regular image to derive depth or shape information for the combined regular image.

6. The camera system of claim 5 , wherein the first frame period is shorter than a sum of the second frame period and the third frame period.

7. The camera system of claim 5 , wherein the structured-light image is captured at lower spatial resolution than the first regular image and the second regular image.

8. The camera system of claim 5 , wherein said one or more analog-to-digital converters correspond to a configurable analog-to-digital converter to provide both the first dynamic range and the second dynamic range.

9. The camera system of claim 5 , wherein said one or more analog-to-digital converters correspond to two analog-to-digital converters with two different resolutions to provide both the first dynamic range and the second dynamic range.

10. An endoscope for in vivo viewing of human gastrointestinal (GI) tract, comprising:
a structured light source;
a non-structured light source;
an integrated image sensor, comprising:
a pixel array being responsive to light energy received by the pixel array to produce pixel signals having a voltage level depending on the light energy received by the pixel array;
one or more output circuits coupled to the pixel array to access the pixel signals produced by the pixel array;
one or more analog-to-digital converters having a first dynamic range and a second dynamic range;
one or more timing and control circuits coupled to the pixel array, said one or more output circuits, said one or more analog-to-digital converters or a combination thereof, wherein said one or more timing and control circuits are arranged to:
capture, by the pixel array, a structured-light image formed on a common image plane during a first frame period by applying first reset signals to the pixel array to

Assignees

• Capso Vision Inc
• Capsovision Inc

Inventors

• Wang Kang-Huai
• Xu Yi
• Wilson Gordon C
• Wu Chenyu

Classifications

• G01B11/2513 Primary

  with several lines being projected in more than one direction, e.g. grids, patterns · CPC title

• H04N25/589 Primary

  with different integration times, e.g. short and long exposures · CPC title

• H04N25/534

  depending on the spectral component · CPC title

• H04N25/59

  by controlling the amount of charge storable in the pixel, e.g. modification of the charge conversion ratio of the floating node capacitance · CPC title

• H04N25/587

  acquired sequentially, e.g. using the combination of odd and even image fields · CPC title