Large Field of View Multi-Camera Endoscopic Apparatus with Omni-Directional Illumination

Patent metadata

Field	Value
Publication number	US-2017013193-A1
Application number	US-201515120527-A
Country	US
Kind code	A1
Filing date	Feb 24, 2015
Priority date	Feb 26, 2014
Publication date	Jan 12, 2017
Grant date	—

Abstract

A multi-camera hemispherical very wide field of view imaging apparatus with omnidirectional illumination capability comprises a cylindrical body ( 4, 4. a, 4. b ), a hemispherical mechanical frame ( 2 ) arranged on one end of the cylindrical body ( 4, 4. a, 4. b ), a plurality of imaging channels ( 3 ), each imaging channel ( 3 ) comprising at least an image sensor and related optics with a fixed focus appropriate for endoscopic imaging, the plurality of imaging channels ( 3 ) being distributed over the hemispherical mechanical frame ( 2 ), a light source arranged centre-down at a back part of the plurality of imaging channels ( 3 ) and inside or at the end of the cylindrical body ( 4, 4. a, 4. b ). Each imaging channel ( 3 ) comprises a plurality of lightning channels ( 1 ) around their centre, each of the plurality of lightning channels ( 1 ) comprising at least one microfiber light guide having a determined angle of curvature arranged to transmit the light from the light source. The imaging apparatus further comprises a control and processing circuit ( 5 ) comprising a camera control unit ( 6 ), an illumination control unit ( 7 ), an illumination unit ( 8 ), a sample and capture unit ( 9 ), an image processing unit ( 10 ) and an output interface ( 11 ) to a PC. The camera control unit ( 6 ) is configured to power each of the plurality of imaging channels ( 3 ) and make automatic gain compensation for each imaging channel ( 3 ), the illumination control unit ( 7 ) is configured for automatic intensity dimming, the sample and capture unit ( 9 ) is an interface circuit for correct sampling, extraction and capturing frames of individual imaging channels ( 3 ), the image processing unit ( 10 ) is configured for constructing a spherical panoramic image by applying a determined algorithm, and the output interface ( 11 ) is arranged to output the spherical panoramic image to a system configured to visualize it.

First claim

1 - 11 . (canceled)

12 . A multi-camera hemispherical very wide field of view imaging apparatus with omnidirectional illumination capability, the apparatus comprising:
a cylindrical body; a hemispherical mechanical frame arranged on an end of the cylindrical body; a plurality of imaging channels, each imaging channel including an image sensor and optics with a fixed focus for endoscopic imaging, the plurality of imaging channels being distributed over the hemispherical mechanical frame; a light source arranged center-down at a back part of the plurality of imaging channels and inside or at the end of the cylindrical body; and a control and processing circuit including a camera control unit, an illumination control unit, an illumination unit, a sample and capture unit, an image processing unit, and an output interface to a system, wherein each imaging channel includes a plurality of lightning channels around a center of the respective imaging channel, each of the plurality of lightning channels including a microfiber light guide having a determined angle of curvature arranged to transmit light from the light source, and wherein the camera control unit is configured to power each of the plurality of imaging channels and to perform automatic gain compensation for each of the plurality of imaging channels, the illumination control unit is configured for automatic intensity dimming, the sample and capture unit having an interface circuit for sampling, extraction and capturing frames of individual imaging channels, the image processing unit is configured generate a spherical panoramic image by applying an algorithm, and the output interface is arranged to output the spherical panoramic image generated by the image processing unit to the system, the system configured to visualize the spherical panoramic image.

13 . A method for distributing imaging channels over a hemispherical mechanical frame of a very wide field of view imaging apparatus, the method comprising the steps of:
defining a first distance that departs from a center of the hemispherical mechanical frame, the first distance smaller than or equal to a second distance at which an object to be viewed is separated from the center of the hemispherical mechanical frame; and generating a virtual hemisphere and performing an equal area tessellation method on the virtual hemisphere with an arbitrary number of imaging directions, a center of the virtual hemisphere corresponding to the center of the hemispherical mechanical frame, and a radius of the virtual hemisphere being equal to the first distance, wherein the equal area tessellation method includes,
dividing a surface of the virtual hemisphere into equal sized surface areas, a number of equal sized surface areas corresponding to the arbitrary number of imaging directions,
starting with the arbitrary number of imaging directions located respectively at a center of a corresponding one of the equal sized surface areas,
iteratively checking if a coverage of the first distance is achieved by adding or subtracting one imaging direction to a tessellation at each iteration,
wherein the iteratively checking includes,
ensuring that an angle of view of a single imaging channel is greater than or equal to a maximum radial distance on the virtual hemisphere, inside any of the equal sized tessellation areas, the angle of view of the imaging channels being assumed as constraints dictated by optics of the imaging channels,
stopping the iterative checking after the coverage of the first distance is achieved,
fixing all the imaging directions used in the last iterative checking as a set of locations, each location corresponding to a center of a tessellation cell on the virtual hemisphere, and
back projecting the locations from the virtual hemisphere to the hemispherical mechanical frame to obtain a distribution of the imaging channels on the hemispherical mechanical frame.

14 . A method for constructing a panoramic image, comprising the steps of:
generating an output as a panoramic Bayer pattern image and converting the panoramic Bayer pattern image to a full-color image, wherein each pixel of the generated panoramic Bayer pattern image represents an intensity value of a color channel of either red, green or blue wavelength, a sequence of the color channels on a Bayer pattern used for the Bayer pattern image is any known Bayer pattern, the intensity value of each pixel being a combination of sampled intensity values of image sensors of imaging channels placed on a hemispherical frame, each combination being based on a selection of candidate intensity values of the image sensors of the imaging channels on the hemispherical mechanical frame, and a Bayer pattern representation of the images is used during the selection of the candidate intensity values for each combination to generate each panoramic pixel intensity value.

15 . The method of claim 14 , wherein any Bayer pattern sequence is accepted as Bayer pattern images from the image sensors.

16 . The method of claim 14 , further comprising the step of:
determining the selection of candidate intensity values for each combination by a trigonometric relation of a final panorama pixel viewing direction and an imaging channel sampling direction.

17 . The method of claim 14 , further comprising the step of:
defining each panorama pixel viewing direction as a vector starting from the center of the hemispherical mechanical frame and directed through a point in the space at infinite proximity.

18 . The method of claim 17 , wherein each panorama pixel viewing direction is unique and not coincident.

19 . The method of claim 14 , further comprising the step of:
determining a viewing direction of each imaging channel by using a placement and orientation of the viewing direction on the hemispherical mechanical frame.

20 . The method of claim 14 , further comprising the step of:
determining precise relative direction vectors of all the imaging channels with respect to the center of the hemispherical mechanical frame by using a calibration process with a calibration checker board.

21 . The method of claim 14 , wherein the panoramic Bayer pattern image is demosaiced after the panoramic Bayer pattern is generated by an image processing device and is converted to a full color image by using an interpolation method by the image processing device.

22 . The method of claim 21 , wherein the interpolation method takes into account object boundaries in the image.

Assignees

• Ecole Polytechnique Fed De Lausanne (Epfl)

Inventors

• Cogal Ömer
• Leblebici Yusuf

Classifications

• H04N23/76

  by influencing the image signals · CPC title

• H04N23/51

  Housings · CPC title

• H04N23/56

  provided with illuminating means · CPC title

• H04N23/72

  Combination of two or more compensation controls · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title