Large Field of View Multi-Camera Endoscopic Apparatus with Omni-Directional Illumination

Patent metadata

Field	Value
Publication number	US-2020154043-A1
Application number	US-201816185130-A
Country	US
Kind code	A1
Filing date	Nov 9, 2018
Priority date	Nov 9, 2018
Publication date	May 14, 2020
Grant date	—

Abstract

A multi-camera hemispherical very wide field of view imaging apparatus with omnidirectional illumination capability comprises a cylindrical body (4, 4.a, 4.b), a hemispherical mechanical frame (2) arranged on one end of the cylindrical body (4, 4.a, 4.b), a plurality of imaging channels (3), each imaging channel (3) comprising at least an image sensor and related optics with a fixed focus appropriate for endoscopic imaging, the plurality of imaging channels (3) being distributed over the hemispherical mechanical frame (2), a light source arranged centre-down at a back part of the plurality of imaging channels (3) and inside or at the end of the cylindrical body (4, 4.a, 4.b). Each imaging channel (3) comprises a plurality of lightning channels (1) around their centre, each of the plurality of lightning channels (1) comprising at least one microfiber light guide having a determined angle of curvature arranged to transmit the light from the light source. The imaging apparatus further comprises a control and processing circuit (5) comprising a camera control unit (6), an illumination control unit (7), an illumination unit (8), a sample and capture unit (9), an image processing unit (10) and an output interface (11) to a PC. The camera control unit (6) is configured to power each of the plurality of imaging channels (3) and make automatic gain compensation for each imaging channel (3), the illumination control unit (7) is configured for automatic intensity dimming, the sample and capture unit (9) is an interface circuit for correct sampling, extraction and capturing frames of individual imaging channels (3), the image processing unit (10) is configured for constructing a spherical panoramic image by applying a determined algorithm, and the output interface (11) is arranged to output the spherical panoramic image to a system configured to visualize it.

First claim

1 - 11 . (canceled)

12 : A method for constructing a panoramic image, comprising the steps of:
generating an output as a panoramic Bayer pattern image and converting the panoramic Bayer pattern image to a full-color image, wherein each pixel of the generated panoramic Bayer pattern image represents an intensity value of a color channel of either red, green or blue wavelength, a sequence of the color channels on a Bayer pattern used for the Bayer pattern image is any known Bayer pattern, the intensity value of each pixel being a combination of sampled intensity values of image sensors of imaging channels placed on a hemispherical frame, each combination being based on a selection of candidate intensity values of the image sensors of the imaging channels on the hemispherical mechanical frame, and a Bayer pattern representation of the images is used during the selection of the candidate intensity values for each combination to generate each panoramic pixel intensity value.

13 : The method of claim 13 , wherein any Bayer pattern sequence is accepted as Bayer pattern images from the image sensors.

14 : The method of claim 12 , further comprising the step of:
determining the selection of candidate intensity values for each combination by a trigonometric relation of a final panorama pixel viewing direction and an imaging channel sampling direction.

15 : The method of claim 12 , further comprising the step of:
defining each panorama pixel viewing direction as a vector starting from the center of the hemispherical mechanical frame and directed through a point in the space at infinite proximity.

16 : The method of claim 15 , wherein each panorama pixel viewing direction is unique and not coincident.

17 : The method of claim 12 , further comprising the step of:
determining a viewing direction of each imaging channel by using a placement and orientation of the viewing direction on the hemispherical mechanical frame.

18 : The method of claim 12 , further comprising the step of:
determining precise relative direction vectors of all the imaging channels with respect to the center of the hemispherical mechanical frame by using a calibration process with a calibration checker board.

19 : The method of claim 12 , wherein the panoramic Bayer pattern image is demosaiced after the panoramic Bayer pattern is generated by an image processing device and is converted to a full color image by using an interpolation method by the image processing device.

20 : The method of claim 19 , wherein the interpolation method takes into account object boundaries in the image.

21 : A method for distributing imaging channels over a hemispherical mechanical frame of a very wide field of view imaging apparatus, the method comprising the steps of:
defining a first distance that departs from a center of the hemispherical mechanical frame, the first distance smaller than or equal to a second distance at which an object to be viewed is separated from the center of the hemispherical mechanical frame; and generating a virtual hemisphere and performing an equal area tessellation method on the virtual hemisphere with an arbitrary number of imaging directions, a center of the virtual hemisphere corresponding to the center of the hemispherical mechanical frame, and a radius of the virtual hemisphere being equal to the first distance, wherein the equal area tessellation method includes,
dividing a surface of the virtual hemisphere into equal sized surface areas, a number of equal sized surface areas corresponding to the arbitrary number of imaging directions,
starting with the arbitrary number of imaging directions located respectively at a center of a corresponding one of the equal sized surface areas,
iteratively checking if a coverage of the first distance is achieved by adding or subtracting one imaging direction to a tessellation at each iteration,
wherein the iteratively checking includes,
ensuring that an angle of view of a single imaging channel is greater than or equal to a maximum radial distance on the virtual hemisphere, inside any of the equal sized tessellation areas, the angle of view of the imaging channels being assumed as constraints dictated by optics of the imaging channels,
stopping the iterative checking after the coverage of the first distance is achieved,
fixing all the imaging directions used in the last iterative checking as a set of locations, each location corresponding to a center of a tessellation cell on the virtual hemisphere, and
back projecting the locations from the virtual hemisphere to the hemispherical mechanical frame to obtain a distribution of the imaging channels on the hemispherical mechanical frame.

Assignees

• Ecole Polytechnique Fed Lausanne Epfl

Inventors

• Cogal Ömer
• Leblebici Yusuf

Classifications

• A61B1/045

  Control thereof · CPC title

• A61B1/0661

  Endoscope light sources · CPC title

• A61B1/00057

  provided with means for testing or calibration · CPC title

• A61B1/00006

  of control signals · CPC title

• H04N9/09

  Electricity · mapped topic