Method for displaying tumor location within endoscopic images

Patent metadata

Field	Value
Publication number	US-11625825-B2
Application number	US-201916718815-A
Country	US
Kind code	B2
Filing date	Dec 18, 2019
Priority date	Jan 30, 2019
Publication date	Apr 11, 2023
Grant date	Apr 11, 2023

Abstract

A method of displaying an area of interest within a surgical site includes modeling a patient's lungs and identifying a location of an area of interest within the model of the patient's lungs. The topography of the surface of the patient's lungs is determined using an endoscope having a first camera, a light source, and a structured light pattern source. Real-time images of the patient's lungs are displayed on a monitor and the real-time images are registered to the model of the patient's lungs using the determined topography of the patient's lungs. A marker indicative of the location of the area of interest is superimposed over the real-time images of the patient's lungs. If the marker falls outside of the field-of view of the endoscope, an arrow is superimposed over the real-time images to indicate the direction in which the marker is located relative to the field of view.

First claim

What is claimed is:

1. A method of displaying an area of interest within a surgical site, comprising:
generating a three-dimensional (3D) model of lungs of a patient from preoperative images;
identifying a location of an area of interest within the preoperative images;
generating collapsed state 3D models of the patient's lungs in a collapsed state;
updating the location of the area of interest in the collapsed state 3D models;
depicting the area of interest within at least one of the collapsed state the 3D models model of the patient's lungs;
determining a topography of a surface of the patients lungs using a first camera, a light source, and a structured light pattern source of an endoscope;
displaying, on a monitor associated with the endoscope, real-time images of the patients lungs captured by a second camera of the endoscope, the second camera having a field of view;
registering the real-time images of the patient's lungs with the at least one collapsed state 3D model of the patients lungs using the determined topography of the surface of the patient's lungs; and
superimposing, over the real-time images of the patient's lungs, a marker indicative of the location of the area of interest within the at least one collapsed state 3D model of the patient's lungs, a displayed position of the marker remaining stationary relative to the displayed real-time images of the patient's lungs as the field of view of the second camera changes.

2. The method according to claim 1 , further comprising displaying, when the area of interest is outside of the field of view of the second camera, information indicative of a direction in which the area of interest is located relative to the field of view of the second camera.

3. The method according to claim 1 , further comprising advancing the endoscope within a body cavity of the patient.

4. The method according to claim 1 , wherein generating the 3D model of the lungs of the patient includes acquiring computed tomography (CT) data of the patient's lungs.

5. The method according to claim 4 , wherein generating & the collapsed state 3D models of the lungs of the patient includes acquiring tissue data of the patient's lungs.

6. The method according to claim 5 , further comprising storing a software application within a memory of a computer, the computer having a processor configured to execute the software application, which when executed, creates the collapsed state 3D models of the patient's lungs based on the CT data and the tissue data.

7. A system for displaying an area of interest within a surgical site, comprising:
a computer having a processor configured to execute a software application, which when executed, creates a three-dimensional (3D) model of a patient's lungs from preoperative images; and
a monitor associated with the computer and connected to an endoscope, the monitor configured to display real-time images of the patients lungs captured by a first camera of the endoscope, the first camera having a field of view,
wherein the processor is configured to determine a topography of the patient's lungs, generate a collapsed state 3D model, register the real-time images of the patients lungs with the collapsed state 3D model of the patients lungs using the determined topography of the a surface of the patient's lungs, and superimpose, over the real-time images of the patients lungs, a marker indicative of the a location of the area of interest within the patients lungs as the area of interest appears in the collapsed state 3D model, a displayed position of the marker remaining stationary relative to the displayed real-time images of the patients lungs as the field of view of the first camera changes.

8. The system according to claim 7 , wherein the computer includes a memory for storing computed tomography (CT) data and tissue data associated with the patient's lungs.

9. The system according to claim 7 , wherein the processor is configured to display, when the area of interest is outside of the field of view of the first camera, information on the monitor indicative of a direction in which the area of interest is located relative to the field of view of the first camera.

10. The system according to claim 7 , wherein the endoscope is configured to be advanced within a body cavity of the patient.

11. The system according to claim 7 , wherein the preoperative images are computed tomography (CT) images of the patient's lungs.

12. The system according to claim 7 , wherein the computer is configured to acquire tissue data of the patient's lungs.

13. The system according to claim 7 , wherein the endoscope includes a second camera.

14. The system according to claim 13 , wherein the endoscope includes a light source.

15. The system according to claim 14 , wherein the endoscope includes a structured light pattern source.

16. The system according to claim 15 , wherein the topography of the patient's lungs is determined using the second camera, the light source, and the structured light pattern source.

17. A method of displaying an area of interest within a surgical site, comprising:
instructing a processor associated with a computer to execute a software application, which when executed, creates a three-dimensional (3D) model of a patient's lungs from preoperative images;
displaying real-time images of the patient's lungs captured by a first camera of an endoscope on a monitor associated with the computer, the first camera having a field of view; and
instructing the processor to:
generate a collapsed state 3D model of the patients lungs;
determine a topography of the patients lungs;
register the real-time images of the patient's lungs with the collapsed state 3D model of the patient's lungs using the determined topography of the patient's lungs; and
superimpose, over the real-time images of the patients lungs, a marker indicative of a the location of the area of interest within the patients lungs as the area of interest appears in the collapsed state 3D model, a displayed position of the marker remaining stationary relative to the displayed real-time images of the patients lungs as the field of view of the first camera changes.

18. The method according to claim 17 , further including displaying, when the area of interest is outside the field of view of the first camera, information on the monitor indicative of a direction in which the area of interest is located relative to the field of view of the first camera.

19. The method according to claim 17 , wherein determining the topography of the patient's lungs includes determining the topography of the patient's lungs using a second camera, a light source, and a structured light pattern source associated with the endoscope.

20. The method according to claim 17 , wherein instructing the processor associated with the computer to execute the software application includes acquiring computed tomography (CT) data and tissue data of the patient's lungs.

Assignees

• Covidien Lp

Inventors

• Cantrall Irena
• Komp John W
• Rossetto Francesca

Classifications

• A61B2034/107

  Visualisation of planned trajectories or target regions · CPC title

• G06T2207/30096

  Tumor; Lesion · CPC title

• A61B1/313

  for introducing through surgical openings, e.g. laparoscopes · CPC title

• G06T7/30

  Determination of transform parameters for the alignment of images, i.e. image registration · CPC title

• A61B34/10 Primary

  Computer-aided planning, simulation or modelling of surgical operations · CPC title