Single image sensor for capturing mixed structured-light images and regular images

Patent metadata

Field	Value
Publication number	US-9936151-B2
Application number	US-201514884788-A
Country	US
Kind code	B2
Filing date	Oct 16, 2015
Priority date	Oct 16, 2015
Publication date	Apr 3, 2018
Grant date	Apr 3, 2018

Abstract

A method and device for capturing a mixed structured-light image and regular image using an integrated image sensor are disclosed, where the structured-light image is captured using a shorter frame period than the regular image. In order to achieve a shorter frame period for the structured-light image, the structured-light image may correspond to an image captured with reduced dynamic range, reduced spatial resolution, or a combination of them. The capturing process comprises applying reset signals to a pixel array to reset rows of pixels of the pixel array, reading-out analog signals from the rows of pixels of the pixel array and converting the analog signals from the rows of pixels of the pixel array into digital outputs for the image using one or more analog-to-digital converters.

First claim

The invention claimed is:

1. A method of capturing images of a scene using a camera comprising an image sensor, the method comprising:
projecting, by a structured light source, a first structured light to a scene in a field of view of the image sensor;
capturing, by the image sensor, a first structured-light image formed on a common image plane during a first frame period by applying first reset signals to the image sensor to reset rows of pixels of the image sensor, exposing the rows of pixels of the image sensor to structured light to cause first analog signals from the rows of pixels and converting the first analog signals from the rows of pixels of the image sensor into first digital outputs to form the first structured-light image using one or more analog-to-digital converters;
capturing, by the image sensor, a regular image formed on a same image plane as the common image plane using the image sensor during a second frame period by applying second reset signals to the image sensor to reset the rows of pixels of the image sensor, exposing the rows of pixels to non-structured light to cause second analog signals from the rows of pixels and converting the second analog signals from the rows of pixels into second digital outputs to form the regular image using said one or more analog-to-digital converters; and
wherein the first frame period is shorter than the second frame period and wherein the first structured-light image is captured before or after the regular image to derive depth or shape information for the regular image.

2. The method of claim 1 , wherein the first structured-light image is captured with a reduced dynamic range of the image sensor with reduced pixel resolution or reduced spatial resolution compared to the regular image.

3. The method of claim 2 , wherein the reduced dynamic range corresponds to reduced resolution of said one or more analog-to-digital converters.

4. The method of claim 2 , wherein the reduced dynamic range corresponds to reduced ramping period for generating a ramp reference voltage, wherein the ramp reference voltage is used by said one or more analog-to-digital converters to compare with an input analog voltage.

5. The method of claim 2 , wherein said one or more analog-to-digital converters correspond to one or more successive-approximation analog-to-digital converters and the reduced dynamic range corresponds to a reduced number of successive approximations for refining a reference voltage supplied to said one or more successive-approximation analog-to-digital converters to compare with an input analog voltage.

6. The method of claim 2 , wherein the reduced dynamic range corresponds to reduced integration time for the image sensor to accumulate electronic charges.

7. The method of claim 6 , wherein gain of the first analog signals from the rows of pixels of the image sensor is increased for the first structured-light image.

8. The method of claim 1 , further comprising projecting the structured light with first intensity onto the scene during a first integration time of the first frame period.

9. The method of claim 8 , wherein a period of the first intensity is substantially shorter than human visual retention time.

10. The method of claim 8 , wherein the structured light is generated using multiple light sources with at least two different colors or patterns.

11. The method of claim 10 , further comprising generating different first control signals from the image sensor to trigger said projecting the structured light for two different structured light sources when said two different structured light sources are used to generate the structured light.

12. The method of claim 8 , wherein first spectrum associated with the structured light is substantially distinct from second spectrum associated with second regular images of an anticipated scene.

13. The method of claim 12 , wherein the image sensor corresponds to a color image sensor comprising at least first pixels for a first color and second pixels for a second color arranged in a mosaic pattern, and the first spectrum is substantially concentrated on one color.

14. The method of claim 13 , wherein the first structured-light image is captured at reduced spatial resolution by reading out only selected digital outputs related to pixels of one color substantially corresponding to the first spectrum.

15. The method of claim 8 , further comprising projecting non-structured light with second intensity onto the scene during a second integration time of the second frame period.

16. The method of claim 15 , further comprising generating from the image sensor a first control signal to trigger said projecting the structured light and generating a second control signal to trigger said projecting the non-structured light.

17. The method of claim 15 , wherein the non-structured light corresponds to broadband.

18. The method of claim 15 , wherein the non-structured light is generated using narrowband illumination or fluoroscopic excitation.

19. The method of claim 15 , wherein the first intensity for the structured light is substantially higher than the second intensity for the non-structured light.

20. The method of claim 1 , wherein a first integration time for the first structured-light image is substantially shorter than a second integration time for the regular image.

21. The method of claim 1 , wherein a first minimum row reset time among the rows of pixels of the image sensor for the first structured-light image is substantially shorter than a second minimum row reset time among the rows of pixels of the image sensor for the regular image.

22. The method of claim 1 , wherein a settling time associated with a reference voltage, provided to said one or more analog-to-digital converters to compare with an input analog voltage, is shorter for the first structured-light image than for the regular image.

23. The method of claim 1 , wherein the structured light is applied during integration period of the image sensor for the first structured-light image, and the structured light starts substantially at the same time as beginning of last row integration time and ends substantially at the same time as beginning of first row readout period.

24. The method of claim 1 , further comprising providing the first structured-light image to derive depth or shape information for the regular image.

25. The method of claim 1 , wherein the first structured-light image is captured at lower spatial resolution than the regular image.

26. The method of claim 25 , wherein the first structured-light image is captured with reduced vertical resolution by only retaining selected rows of pixels and skipping remaining rows of pixels of the image sensor.

27. The method of claim 1 , wherein the first structured-light image is captured with a reduced image area in a vertical direction, horizontal direction or both compared to the regular image.

28. The method of claim 1 , wherein a following first structured-light image and a following regular image are generated and provided repeatedly to form a first structured-light image sequence and a regular image sequence, wherein the first structured-light image sequence is used for deriving depth or shape information for a regular image sequence for viewing.

29. The method of claim 1 further comprising:
capturing, by the image sensor, a second structu

Assignees

• Capso Vision Inc
• Capsovision Inc

Inventors

• Wang Kang-Huai
• Xu Yi
• Wilson Gordon C
• Wu Chenyu

Classifications

• H04N25/587

  acquired sequentially, e.g. using the combination of odd and even image fields · CPC title

• H04N25/534

  depending on the spectral component · CPC title

• G01B11/2513 Primary

  with several lines being projected in more than one direction, e.g. grids, patterns · CPC title

• H04N25/70

  SSIS architectures; Circuits associated therewith · CPC title

• H04N23/555

  for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes · CPC title