Method and apparatus to provide updated patient images during robotic surgery

Patent metadata

Field	Value
Publication number	US-10376335-B2
Application number	US-201515575603-A
Country	US
Kind code	B2
Filing date	Sep 10, 2015
Priority date	May 20, 2015
Publication date	Aug 13, 2019
Grant date	Aug 13, 2019

Abstract

In a method and an apparatus to provide updated images during a robotically-implemented surgical procedure, 3D data is obtained of a volume of a patient, which includes anatomy involved in the procedure. The anatomy is segmented from a reconstructed image of the volume. During the procedure, the surgeon applies forces on the anatomy, causing a geometric change of the anatomy. Force sensors in the surgical robot detect these forces, which are supplied to a processor that controls display of the segmented anatomy at a display screen. From the applied forces and the physical properties of the anatomy, the processor calculates the geometric change of the anatomy that has occurred and modifies the appearance and/or position of the displayed segmented anatomy on the display screen in real time during the procedure, so as to visualize the geometric change.

First claim

We claim as our invention:

1. A method for providing current images of a surgical site during a surgical procedure, the method comprising:
receiving, by an imaging system, three-dimensional (3D) data representing a volume of a patient on a patient table from a medical imaging device of the imaging system, the volume comprising an anatomy to be manipulated, during a surgical procedure, by a surgical robot;
constructing, by the imaging system, a 3D image of the volume from the 3D data;
constructing, by the imaging system, a segmented 3D volumetric image of the anatomy from the 3D image of the volume;
bringing, by the imaging system, a coordinate system of the surgical robot into registration with a coordinate system of the medical imaging device;
receiving, by the imaging system, a signal representing at least one force applied to the anatomy during the surgical procedure and detected by a force sensor of the surgical robot, wherein the at least one force produces a geometrical change of the anatomy;
automatically updating, by the imaging system, the segmented 3D volumetric image of the anatomy, based on the at least one force, to produce an updated segmented 3D volumetric image of the anatomy that visually shows the geometrical change of the anatomy; and
displaying, by a display screen of the imaging system, the updated segmented 3D volumetric image of the anatomy in real-time during the surgical procedure.

2. A method as claimed in claim 1 , further comprising:
storing at least one material property respectively for different anatomical objects in an electronic database, wherein the at least one material property is a same or different material property for each anatomical object, and
accessing the electronic database from the processor to obtain the at least one material property from the database for the anatomy in the segmented 3D volumetric image.

3. A method as claimed in claim 2 , further comprising:
obtaining a pre-operative data set of the patient prior to receiving the 3D data of the patient on the table, the pre-operative data comprising the volume of the patient that is comprised in the 3D data;
providing the pre-operative data set to the processor;
bringing, by the processor, the pre-operative data set into registration with the 3D image constructed from the 3D data and then combining the pre-operative data set with the 3D image constructed from the 3D data to generate a combined image; and
segmenting the anatomy from the combined image to obtain the segmented 3D volumetric image of the anatomy.

4. A method as claimed in claim 1 , further comprising:
obtaining an endoscopic image of the anatomy and displaying the endoscopic image of the anatomy at the display screen or a second display screen during the surgical procedure; and
during the surgical procedure, manipulating the updated segmented 3D volumetric image of the anatomy to show a view of the anatomy that is not visible in the endoscopic image.

5. A method as claimed in claim 1 , further comprising:
segmenting the anatomy from the 3D image of the volume constructed from the 3D data by interaction, via the processor, with the 3D image of the volume constructed from the 3D data.

6. A method as claimed in claim 1 , further comprising:
automatically segmenting the anatomy, by the processor, from the 3D image of the volume constructed from the 3D data, by executing a segmentation algorithm with pattern recognition.

7. An apparatus for providing current images of a surgical site during a surgical procedure, the apparatus comprising:
a control computer having at least one processor, the control computer configured to operate a medical imaging device to acquire three-dimensional (3D) data representing a volume of a patient on a patient table, the volume comprising anatomy to be manipulated during a surgical procedure by a surgical robot; and
a display screen in communication with the at least one processor of the control computer,
wherein the at least one processor of the control computer is configured to construct a 3D image of the volume from the 3D data,
wherein the at least one processor is configured to construct a segmented 3D volumetric image of the anatomy from the 3D image of the volume,
wherein the at least one processor is configured to bring a coordinate system of the surgical robot into registration with a coordinate system of the medical imaging device,
wherein the at least one processor is configured to receive an electronic signal representing at least one force applied to the anatomy during the surgical procedure and detected by a force sensor of the surgical robot, wherein the at least one force produces a geometrical change of the anatomy,
wherein the at least one processor is configured to automatically update the segmented 3D volumetric image of the anatomy, based on the at least one force, to produce an updated segmented 3D volumetric image of the anatomy that visually shows the geometrical change of the anatomy, and
wherein the at least one processor is configured to display the updated segmented 3D volumetric image of the anatomy in real-time during the surgical procedure at the display screen.

8. An apparatus as claimed in claim 7 , further comprising:
an electronic database in which at least one material property respectively for different anatomical objects is stored, wherein the at least one material property is a same or different material property for each anatomical object, and
wherein the processor is configured to access the electronic database to obtain the at least one material property from the database for the anatomy in the segmented 3D volumetric image.

9. An apparatus as claimed in claim 8 , further comprising:
a source of a pre-operative data set acquired from the patient prior to receiving the 3D data of the patient on the table, the preoperative data comprising the volume of the patient that is comprised in the 3D data;
wherein the processor is configured to receive the pre-operative data set and to bring the pre-operative data set into registration with the 3D image constructed from the 3D data, and then combine the pre-operative data set with the 3D image constructed from the 3D data, and
wherein the processor is configured to segment the anatomy from the combined image to obtain the segmented 3volumetric image of the anatomy.

10. An apparatus as claimed in claim 7 , further comprising:
an endoscopic camera at the surgical robot, wherein the endoscopic camera is configured to obtain an endoscopic image of the anatomy and display the endoscopic image of the anatomy at the display screen or a second display screen during the surgical procedure,
wherein the control computer is configured to allow, during the surgical procedure, a manipulation of the updated segmented 3D volumetric image of the anatomy to show a view of the anatomy that is not visible in the endoscopic image.

11. An apparatus as claimed in claim 7 , further comprising:
a user interface configured to allow segmenting of the anatomy from the 3D image of the volume constructed from the 3D data by interaction with the at least one processor, via the interface, with the 3D image of the volume constructed from the 3D data.

12. An apparatus as claimed in claim 7 , wherein the at least one processor is configured to automatically segment the anatomy from the 3D image of the volume constructed from the 3D data, by executing a segmentation algorithm with pattern recognition.

13. A non-transitory, computer-readable data storage medium encoded with programming instructions, the storage medium being loaded into a computer of a surgery facility comprising a pati

Assignees

• Siemens Healthcare Gmbh
• Intuitive Surgical Operations
• Intuitive Surgical Operatons Inc

Inventors

• Azizian Mahdi
• Blohm Lutz
• Kunze Holger
• Niebler Christine
• Sorger Jonathan

Classifications

• A61B34/30

  Surgical robots · CPC title

• A61B2090/367

  creating a 3D dataset from 2D images using position information · CPC title

• A61B2090/368

  changing the image on a display according to the operator's position · CPC title

• A61B2090/364

  Correlation of different images or relation of image positions in respect to the body · CPC title

• A61B90/37 Primary

  Surgical systems with images on a monitor during operation · CPC title