Endoscope with dual image sensors

Patent metadata

Field	Value
Publication number	US-10955657-B2
Application number	US-201916695500-A
Country	US
Kind code	B2
Filing date	Nov 26, 2019
Priority date	Dec 20, 2018
Publication date	Mar 23, 2021
Grant date	Mar 23, 2021

Abstract

An endoscopic camera has two or more cameras positioned at a distal end, positioned to provide partially overlapping fields of view. The cameras communicate captured digital images the length of the flexible endoscope, where they may be saved and processed to provide additional imaging features. With partially overlapping images from two or more cameras, image processing can provide panoramic images, super resolution images, and three-dimensional images. One or more of these image modes may also be enhanced to provide a virtualization window that displays a portion of a larger or higher resolution image. The virtualization window displays the selected area of the image, and may be moved or zoomed around the image to provide a virtual endoscope repositioning experience, where the endoscope remains statically positioned but the virtualization window presents a sense of movement and navigation around the surgical area.

First claim

We claim:

1. A system comprising:
(a) an endoscope comprising:
(i) a coupling unit,
(ii) a shaft extending from the coupling unit, and
(iii) a set of cameras positioned at a distal end of the shaft, wherein the set of cameras are operable to capture and communicate image data to the coupling unit;

(b) an image processor configured to receive image data from the coupling unit; and
(c) a display;
wherein the image processor is configured to:
(i) receive a first image from a first camera of the set of cameras and receive a second image from a second camera of the set of cameras,
(ii) determine a first image unique portion, an overlapping portion, and a second image unique portion based on the first image and the second image, wherein the first image unique portion and the second image unique portion are each defined as not overlapping with another image portion,
(iii) create an enhanced image based on the first image unique portion, the overlapping portion, and the second image unique portion, and
(iv) provide a visualization window that displays a partial section of the enhanced image via the display, wherein the partial section shows a greater level of image detail than portions of the enhanced image not forming a part of the partial section.

2. The system of claim 1 , wherein the image processor is further configured to, when creating the enhanced image, combine the first image unique portion, the overlapping portion, and the second image unique portion to create a panoramic image, wherein the panoramic image comprises a higher resolution than the first image and the second image.

3. The system of claim 1 , wherein the image processor is further configured to, when creating the enhanced image:
(i) determine that the first image and the second image perfectly or substantially overlap, such that the first image unique portion and the second image unique portion contain no image data or negligible image data, and
(ii) combine the overlapping portion from the first image and the second image to create a super resolution image, wherein the super resolution image comprises a higher resolution than the overlapping portion.

4. The system of claim 1 , wherein the image processor is further configured to, when creating the enhanced image, create a 3D image of the overlapping portion, wherein the 3D image is configured to be displayed as one or more of:
(i) an image that is viewable three-dimensionally on a three-dimensional display, and
(ii) an image that is viewable three-dimensionally on a two-dimensional display by pivoting back and forth between a first perspective of the overlapping portion and a second perspective of the overlapping portion.

5. The system of claim 1 , wherein the first camera and the second camera are oriented such that the first camera and the second camera have respective lines of sight that are parallel with each other, and wherein the processor is further configured to determine the overlapping portion based on the first image, the second image, and a static overlap value, wherein the static overlap value is determined based upon a distance between the position of the first camera and the second camera on the distal tip.

6. The system of claim 1 , further comprising a set of operating controls, wherein the image processor is further configured to:
(i) receive a set of inputs via the set of operating controls, and
(ii) update the partial section displayed in the visualization window based on the set of inputs.

7. The system of claim 6 , wherein the image processor is further configured to, where the set of inputs comprises an input indicating a movement of the visualization window along one or more of the x-axis and the y-axis:
(i) determine a destination section of the enhanced image based on the input and the partial section, and
(ii) refresh and provide the visualization window using the destination section as the partial section.

8. The system of claim 7 , wherein the image processor is further configured to, when refreshing and providing the visualization window using the destination section as the partial section:
(i) determine one or more transition sections of the enhanced image located between the partial section and the destination section, and
(ii) display the one or more transition sections in an ordered sequence prior to displaying the destination section.

9. The system of claim 6 , wherein the image processor is further configured to, where the set of inputs comprises an input indicating a movement of the visualization window along the z-axis:
(i) determine a new magnification level based on the input, and
(ii) refresh and redisplay the visualization window based on the partial section and the new magnification level.

10. The system of claim 9 , wherein the image processor is further configured to, when refreshing and providing the visualization window based on the partial section and the new magnification level:
(i) determine one or more transition magnifications of the enhanced image located between the partial section at an original magnification level and the partial section at the new magnification level, and
(ii) display the one or more transition magnifications in an ordered sequence prior to displaying the partial section at the new magnification level.

11. The system of claim 6 , wherein the image processor is further configured to, where the set of inputs comprises an input indicating a rotation of a perspective of the visualization window:
(i) determine a destination perspective based on the input,
(ii) digitally alter the partial section of the enhanced image based on the destination perspective, and
(iii) refresh and provide the visualization window based on the digitally altered partial section.

12. The system of claim 11 , wherein the processor is further configured to, when creating the enhanced image, combine the first image and the second image to produce a third image, wherein the third image comprises a higher resolution than the first image and the second image.

13. The system of claim 1 , wherein each camera of the set of cameras comprises a digital imaging device, wherein each camera of the set of cameras is configured to provide image data to the coupling unit via a conductive connection within the shaft, and wherein the image processor is a processor of an image guided surgery navigation system.

14. A method comprising:
(a) positioning an endoscope at a surgical site, wherein the endoscope comprises a first camera and a second camera positioned at a distal end of a shaft of the endoscope;
(b) at an image processor, receiving a first image from the first camera and a second image from the second camera, wherein the first and second images are captured simultaneously;
(c) determining a first image unique portion, an overlapping portion, and a second image unique portion based on the first image and the second image, wherein the first image unique portion and the second image unique portion are each defined as not overlapping with another image portion;
(d) creating an enhanced image based on the first image unique portion, the overlapping portion, and the second image unique portion;
(e) providing a visualization window that displays a partial section of the enhanced image via a display; and
(f) receiving a set of inputs via a set of operating controls, wherein an input of the set of inputs indicates a movement of the visualization window.

15. The method of claim 14 , further comprising, when creating the enhanced image, combining the first image unique portion, the overlapping portion, and the second image unique po

Assignees

• Acclarent Inc
• Biosense Webster Israel Ltd

Inventors

• Shameli Ehsan
• Algawi Yehuda
• Govari Assaf
• Ebrahimi Babak
• Akbarian Fatemeh
• Palushi Jetmir
• Levi Moran
• Racheli Noam

Classifications

• G02B23/2415 Primary

  Stereoscopic endoscopes · CPC title

• A61B1/000094 Primary

  extracting biological structures · CPC title

• A61B1/00194

  adapted for three-dimensional imaging · CPC title

• A61B1/000095

  for image enhancement · CPC title

• G02B23/243

  Objectives for endoscopes · CPC title