Single image sensor control for capturing mixed mode images

Patent metadata

Field	Value
Publication number	US-10201266-B2
Application number	US-201715613205-A
Country	US
Kind code	B2
Filing date	Jun 3, 2017
Priority date	Oct 16, 2015
Publication date	Feb 12, 2019
Grant date	Feb 12, 2019

Abstract

A method and apparatus for capturing an image sequence using a capsule camera are disclosed. According to the present invention, a first energy-based frame time for the special images is determined based on first light energy perceived by the image sensor and a second energy-based frame time for the regular images is determined based on second light energy perceived by the image sensor. The capsule camera captures the image sequence comprising one or more sets of mixed-type images by configuring the capsule camera to cause a mixed-frame distance between the first energy-based frame time for one special image in a target set of mixed-type images and the second energy-based frame time for one regular image in the target set of mixed-type images smaller than an average frame period.

First claim

The invention claimed is:

1. A method of capturing an image sequence using a capsule camera when the capsule camera travels through a human gastrointestinal tract after the capsule camera is swallowed by a patient, wherein the capsule camera comprises an image sensor and is capable of capturing at least two mixed-type images corresponding to special images and regular images, the method comprising:
determining one first energy-based frame time for the special images based on first light energy perceived by the image sensor;
determining one second energy-based frame time for the regular images based on second light energy perceived by the image sensor;
capturing the image sequence comprising one or more sets of mixed-type images by configuring the capsule camera to cause a mixed-frame time distance between said one first energy-based frame time for one special image in a target set of mixed-type images and said one second energy-based frame time for one regular image in the target set of mixed-type images smaller than an overall average frame period, wherein each set of mixed-type images comprises at least one special image and one regular image; and
providing or outputting the image sequence captured for further processing or viewing; and
wherein the special images are used to control capturing the regular images or used to derive or provide associated information for the regular images; and
wherein an overall average frame rate is substantially lower than a capable frame rate of the capsule camera, and wherein the overall average frame rate is calculated as a sum of individual frame rates and each individual frame rate is associated with one type of said least two mixed-type images.

2. The method of claim 1 , wherein the first light energy perceived by the image sensor is determined by accumulating first illumination power from a light source over a first integration period of the image sensor for the special images and the second light energy perceived by the image sensor is determined by accumulating second illumination power from the light source over a second integration period of the image sensor for the regular images.

3. The method of claim 1 , wherein said one first energy-based frame time is determined at a first time instance when the first light energy exceeds a first energy threshold for a target special image and said one second energy-based frame time is determined at a second time instance when the second light energy exceeds a second energy threshold for a target regular image.

4. The method of claim 3 , wherein the first energy threshold is set to a half of total accumulated first light energy over a first integration period of the image sensor and the second energy threshold is set to a half of total accumulated second light energy over a second integration period of the image sensor.

5. The method of claim 1 , wherein the mixed-frame time distance is smaller than 90 percent of an overall average frame period corresponding to an inverse of the overall average frame rate.

6. The method of claim 1 , wherein the special images are captured with a lower quality.

7. The method of claim 6 , wherein the lower quality corresponds to lower image dynamic range, lower spatial resolution, smaller image size, or a combination thereof.

8. The method of claim 1 , wherein the special images correspond to structured-light images (SLIs) used to derive depth information for associated regular images.

9. The method of claim 8 , wherein each set of mixed-type images consists of one structured-light image (SLI) and one regular image.

10. The method of claim 9 , wherein said one structured-light image (SLI) is captured prior to said one regular image.

11. The method of claim 9 , wherein said one structured-light image (SLI) is captured after said one regular image.

12. The method of claim 8 , wherein each set of mixed-type images consists of one structured-light image (SLI) and two regular sub-images, and wherein the structured-light image is captured between the two regular sub-images and the mixed-frame time distance is derived based on two mixed-frame time distances associated with the structured-light image and the two regular sub-images respectively.

13. The method of claim 12 , wherein the two regular sub-images are combined to form one regular image.

14. The method of claim 1 , wherein the special images correspond to monitor-mode images used to control capturing the regular images.

15. The method of claim 14 , wherein each set of mixed-type images consists of one monitor-mode image and one or more regular images.

16. The method of claim 14 , wherein an activity metric is determined between one monitor-mode image and one previously captured image, and when the activity metric exceeds a threshold, one or more regular images are captured to form one set of mixed-types of images along with said one monitor-mode image.

17. The method of claim 16 , wherein when the activity metric does not exceed the threshold, a next monitor-mode image is captured.

18. The method of claim 1 , wherein the special images correspond to narrow-band images and the regular images correspond to wide-band images.

19. The method of claim 18 , wherein each set of mixed-type images consists of one wide-band image and narrow-band images.

20. An apparatus for capturing an image sequence using a capsule camera when the capsule camera travels through a human gastrointestinal tract after the capsule camera is swallowed by a patient, wherein the capsule camera comprises an image sensor and is capable of capturing at least two mixed-type images corresponding to special images and regular images, the apparatus comprising one or more electronic circuits or processors arranged to:
determining one first energy-based frame time for the special images based on first light energy perceived by the image sensor;
determine one second energy-based frame time for the regular images based on second light energy perceived by the image sensor;
capture the image sequence comprising one or more sets of mixed-type images by configuring the capsule camera to cause a mixed-frame time distance between said one first energy-based frame time for one special image in a target set of mixed-type images and said one second energy-based frame time for one regular image in the target set of mixed-type images smaller than an overall average frame period, wherein each set of mixed-type images comprises at least one special image and one regular image; and
provide or output the image sequence captured for further processing or viewing; and
wherein the special images are used to control capturing the regular images or used to derive or provide associated information for the regular images; and
wherein an overall average frame rate is substantially lower than a capable frame rate of the capsule camera, and wherein the overall average frame rate is calculated as a sum of individual frame rates and each individual frame rate is associated with one type of said least two mixed-type images.

Assignees

• Capsovision Inc

Inventors

• Wang Kang-Huai
• Hadley Mark
• Lee Chung-Ta
• Lin Chang-Wei

Classifications

• H04N23/555

  for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes · CPC title

• H04N25/589

  with different integration times, e.g. short and long exposures · CPC title

• H04N23/667

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title

• H04N25/531

  by controlling rolling shutters in CMOS SSIS · CPC title

• H04N23/56

  provided with illuminating means · CPC title