Endoscopic imaging with augmented parallax

Patent metadata

Field	Value
Publication number	US-11793390-B2
Application number	US-202217839064-A
Country	US
Kind code	B2
Filing date	Jun 13, 2022
Priority date	Jan 10, 2019
Publication date	Oct 24, 2023
Grant date	Oct 24, 2023

Abstract

A method for imaging involves scanning an anatomical object within a patient and capturing reflected IR light with a plurality of cameras that are separate from the scanner. The IR images captured by the IR cameras are associated together to create an integrated image based on parallax between the IR cameras and the scanner. The integrated image is associated with a separate or optical light image of the anatomical object to generate an intra-operative 3D image that can be created in real-time. Systems for effectuating such imaging may include multiple surgical instruments supporting various cameras positioned to capture different fields of view and to increase parallax.

First claim

What is claimed is:

1. A method of imaging within a body of a patient, the method comprising:
generating IR light images of IR light projected by a scanner and reflected off of an anatomical object, a first IR light image captured by a first IR camera coupled to a first surgical instrument and a second IR light image captured by a second IR camera coupled to a second surgical instrument, the first and second surgical instruments separate from the scanner such that the first and second IR cameras are positioned in 3D spaced relationship with each other and the scanner projecting the IR light, wherein the first and second IR light images are captured at a default zoom mode;
determining parallax between the first and second IR cameras and the scanner, wherein the parallax is a distance and angle between the first and second IR cameras and the scanner;
associating the IR light images to create an integrated IR light image based upon the determined parallax;
aligning an optical light image of the anatomical object captured with an optical light camera with the integrated IR light image by adjusting pixels to account for the parallax between the optical light camera and the first and second IR cameras; and
associating the integrated IR light image with the optical light image of the anatomical object to generate a real-time, intra-operative 3D image of the anatomical object.

2. The method according to claim 1 , wherein associating the IR light images include generating a 3D point cloud.

3. The method according to claim 1 , further comprising a third IR camera, the first, second, and third IR cameras arrayed in the body of the patient and capturing IR light reflected off of the anatomical object includes capturing IR light with three IR cameras, each IR camera having a different field of view.

4. The method according to claim 3 , further comprising overlapping the different fields of view.

5. The method according to claim 1 , wherein capturing IR light reflected off of the anatomical object includes capturing IR light reflected off of the anatomical object within a first field of view and a second field of view, the first field of view associated with the first IR camera, the second field of view associated with the second IR camera.

6. The method according to claim 5 , further comprising overlapping the first and second fields of view.

7. The method according to claim 1 , further comprising projecting IR light, with a second scanner, onto the anatomical object within the patient at a different frequency, interleaved timing, or combinations thereof, then that of the IR light projected by the scanner.

8. The method according to claim 1 , wherein associating the integrated IR light image with the optical light imaged of the anatomical object to generate a real-time, intra-operative 3D image of the anatomical object is effectuated in real-time and updated as new optical light images are captured by the optical light camera.

9. The method according to claim 2 , wherein generating the 3D point cloud includes generating the 3D point cloud based on the integrated IR light image.

10. The method according to claim 1 , further comprising warping the optical light image into the integrated IR image.

11. A method of imaging within a body of a patient, the method comprising:
generating IR light images of IR light projected by a scanner and reflected off of an anatomical object, a first IR light image captured by a first IR camera coupled to a first surgical instrument and a second IR light image captured by a second IR camera coupled to a second surgical instrument, the first and second surgical instruments separate from the scanner such that the first and second IR cameras are positioned in 3D spaced relationship with each other and the scanner projecting the IR light, wherein the first and second IR light images are captured at a default zoom mode;
determining parallax between the first and second IR cameras and the scanner, wherein the parallax is a distance and angle between the first and second IR cameras and the scanner;
associating the IR light images to create an integrated IR light image based upon the determined parallax;
aligning an optical light image of the anatomical object captured with an optical light camera with the integrated IR light image by adjusting pixels to account for the parallax between the optical light camera and the first and second IR cameras;
associating the integrated IR light image with the optical light image of the anatomical object to generate a real-time, intra-operative 3D image of the anatomical object; and
displaying the generated real-time, intra-operating 3D image on a display.

12. The method according to claim 11 , wherein associating the IR light images includes generating a 3D point cloud.

13. The method according to claim 11 , further comprising a third IR camera, the first, second, and third IR cameras arrayed in the body of the patient and capturing IR light reflected off of the anatomical object includes capturing IR light with three IR cameras, each IR camera having a different field of view.

14. The method according to claim 13 , further comprising overlapping the different fields of view.

15. The method according to claim 11 , wherein capturing IR light reflected off of the anatomical object includes capturing IR light reflected off of the anatomical object within a first field of view and a second field of view, the first field of view associated with the first IR camera, the second field of view associated with the second IR camera.

16. A method of imaging within a body of a patient, the method comprising:
generating IR light images of IR light projected by a scanner and reflected off of an anatomical object, a first IR light images captured by a first IR camera coupled to a first surgical instrument and a second IR light image captured by a second IR camera coupled to a second surgical instrument, the first and second surgical instruments separate from the scanner such that the first and second IR cameras are positioned in 3D spaced relationship with each other and the scanner projecting IR light, wherein the first and second IR light images are captured at a default zoom mode;
determining parallax between the first and second IR cameras and the scanner, wherein the parallax is a distance and angle between the first and second IR cameras and the scanner;
associating the IR light images to create an integrated IR light image based upon the determined parallax;
aligning an optical light image of the anatomical object captured with an optical light camera with the integrated IR light image by adjusting pixels to account for the parallax between the optical light camera and the first and second IR cameras; and
associating the integrated IR light image with the optical light image of the anatomical object in real time and updated as new optical light images are captured by the optical light camera to generate a real-time, intra-operative 3D image of the anatomical object.

17. The method according to claim 16 , wherein associating the IR light images includes generating a 3D point cloud.

18. The method according to claim 16 , further comprising a third IR camera, the first, second, and third IR cameras arrayed in the body of the patient and capturing IR light reflected off of the anatomical object includes capturing IR light with three IR cameras, each IR camera having a different field of view.

19. The method according to claim 18 , further comprising overlapping the different fields of view.

Assignees

• Covidien Lp

Inventors

• Komp John W

Classifications

• A61B1/0005 Primary

  combining images e.g. side-by-side, superimposed or tiled · CPC title

• A61B1/000094

  extracting biological structures · CPC title

• A61B1/00194

  adapted for three-dimensional imaging · CPC title

• A61B1/046

  for infrared imaging · CPC title

• A61B90/37

  Surgical systems with images on a monitor during operation · CPC title