System and method for generating a three-dimensional model of a surgical site

Patent metadata

Field	Value
Publication number	US-11793402-B2
Application number	US-202117329029-A
Country	US
Kind code	B2
Filing date	May 24, 2021
Priority date	Dec 10, 2018
Publication date	Oct 24, 2023
Grant date	Oct 24, 2023

Abstract

A system for generating a 3D model of a surgical site includes a 3D endoscope and a computing device coupled to the 3D endoscope. The 3D endoscope includes a scanner for scanning a surface of a surgical site and a camera source for generating images of the surgical site. A 3D model of the surgical site, including objects therein, is generated using scan data and image data. The 3D model is updated by detecting a change in the surgical site, isolating a region of the surgical site where the change is detected, generating second scan data by scanning the surface of the isolated region, and updating the 3D model generated using the second scan data of the surface of the isolated region.

First claim

What is claimed is:

1. A method for generating a 3D model comprising:
continuously scanning a surface of a surgical site using a structured light scanner embodied on an endoscope;
continuously capturing images of a scan pattern projected on the surgical site by the structured light scanner;
generating a 3D model of the surgical site based on the captured images of the scan pattern at the based on the captured scan pattern;
detecting a change in the surgical site during the continuous scanning from a source separate from the endoscope;
isolating a region of the surgical site where the change is detected;
flattening the isolated region in the 3D model;
capturing images of the scan pattern in the isolation region of the surgical site; and
updating the 3D model generated using the captured images of the scan pattern in the isolated regions of the surgical site.

2. The method of claim 1 , wherein source separate from the endoscope is a motion sensor on a patient or a separate camera.

3. The method of claim 1 , wherein detecting the change in the surgical site includes:
comparing an image of the scan pattern captured a time t with an image of the scan pattern captured at a time t−1; and
determining that there exists the change in the surgical site when the image of the scan pattern captured at time t is different from the image of the scan pattern captured at time t−1.

4. The method of claim 1 further comprising capturing optical images of the surgical site.

5. The method of claim 4 , wherein generating the 3D model of the surgical site includes warping the optical images onto the images of the scan pattern.

6. The method of claim 4 , wherein the optical images and the images of the scan pattern are simultaneously captured.

7. The method of claim 6 , wherein further comprising correlation of the optical images with preoperative images.

8. A method for generating a 3D model comprising:
continuously scanning a surface of a surgical site using a structured light scanner embodied on an endoscope;
continuously capturing images of a scan pattern projected on the surgical site by the structured light scanner;
generating a 3D model of the surgical site based on the captured images of the scan pattern at the based on the captured scan pattern;
detecting a change in the surgical site during the continuous scanning;
isolating a region of the surgical site where the change is detected;
flattening the isolated region in the 3D model;
capturing images of the scan pattern in just the isolation region of the surgical site; and
updating the 3D model generated using the captured images of the scan pattern in just the isolated regions of the surgical site.

9. The method of claim 8 , wherein detecting the change in the surgical site includes detecting movement of an object in the surgical site.

10. The method of claim 8 , wherein detecting the change in the surgical site includes:
comparing an image of the scan pattern captured a time t with an image of the scan pattern captured at a time t−1; and
determining that there exists the change in the surgical site when the image of the scan pattern captured at time t is different from the image of the scan pattern captured at time t−1.

11. The method of claim 8 further comprising capturing optical images of the surgical site.

12. The method of claim 11 , wherein generating the 3D model of the surgical site includes warping the optical images onto the images of the scan pattern.

13. The method of claim 11 , wherein the optical images and the images of the scan pattern are simultaneously captured.

14. The method of claim 13 , wherein further comprising correlation of the optical images with preoperative images.

15. A method for generating a 3D model comprising:
continuously scanning a surface of a surgical site using a structured light scanner embodied on an endoscope;
continuously capturing images of a scan pattern projected on the surgical site by the structured light scanner;
generating a 3D model of the surgical site based on the captured images of the scan pattern at the based on the captured scan pattern;
detecting a change in the surgical site during the continuous scanning;
isolating a region of the surgical site where the change is detected;
flattening the isolated region in the 3D model;
capturing images of the scan pattern in just the isolation region of the surgical site; and
updating the 3D model generated using the captured images of the scan pattern in just the isolated regions of the surgical site, wherein when the detected change is a movement of a surgical tool through the scan pattern projected on the surgical site, the change is ignored and the continuously scanning and continuously capturing images continues.

16. The method of claim 15 , wherein detecting the change in the surgical site includes detecting movement of an object in the surgical site.

17. The method of claim 15 , wherein detecting the change in the surgical site includes:
comparing an image of the scan pattern captured a time t with an image of the scan pattern captured at a time t−1; and
determining that there exists the change in the surgical site when the image of the scan pattern captured at time t is different from the image of the scan pattern captured at time t−1.

18. The method of claim 15 further comprising capturing optical images of the surgical site.

19. The method of claim 18 , wherein generating the 3D model of the surgical site includes warping the optical images onto the images of the scan pattern.

20. The method of claim 18 , wherein the optical images and the images of the scan pattern are simultaneously captured.

Assignees

• Covidien Lp

Inventors

• Komp John W

Classifications

• A61B1/3132 Primary

  for laparoscopy · CPC title

• A61B1/000094

  extracting biological structures · CPC title

• A61B1/00172

  with means for scanning · CPC title

• A61B1/00194

  adapted for three-dimensional imaging · CPC title

• G06T7/11

  Region-based segmentation · CPC title